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PREFACE 


This  document  was  prepared  by  the  Office  of  Systems  Research 
and  Analysis,  Transportation  Systems  center,  under  the 
sponsorship  of  the  office  of  Service  and  Methods 
Demonstrations,  Urban  Mass  Transportation  Administration. 

It  consists  of  evaluation  guidelines  for  use  by  contractors 
responsible  for  evaluating  Service  and  Methods  Demonstration 
projects.  Although  these  guidelines  were  prepared 
specifically  for  the  Service  and  Methods  Demonstration 
Program,  PPA  UM-527,  their  potential  applicability  extends 
beyond  the  evaluation  of  UMTA-sponsored  transit 
demonstration  projects  to  the  evaluation  of  any  type  of 
transit  innovation. 

The  contents  of  this  document  reflect  comments  from  various 
reviewers  on  two  previous  draft  versions  as  well  as  several 
months  of  TSC/contractor  experience  using  the  guidelines. 

It  is  anticipated  that  this  document  will  be  modified 
periodically  to  reflect  additional  experience  gained  in 
evaluating  Service  and  Methods  Demonstration  projects. 

The  authors  wish  to  give  special  acknowledgement  to  Peter 
Mengert  and  Mary  Stearns  for  their  extensive  contributions 
to  the  statistical  methodology  and  survey  methodology 
appendices.  In  addition,  the  authors  wish  to  acknowledge 
the  valuable  review  comments  provided  by  persons  at  UMTA, 
TSC,  the  Institute  of  Public  Administration,  the  Urban 
Institute,  and  the  four  contractor  organizations  (Crain  and 
Associates,  CACI,  Inc.,  ECI  Systems,  Inc.,  and  SYSTAN, 
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CHAPTER  I 


OVERVIEW  OF  EVALUATION  GUIDELINES 


This  manual  presents  guidelines  for  planning, 
implementing,  and  reporting  the  findings  of  the  evaluation 
of  Urban  Mass  Transportation  Administration  (UMTA)  Service 
and  Methods  Demonstration  (SMD)  projects.  These  evaluation 
guidelines  are  intended  for  use  by  organizations  engaged  by 
the  Transportation  Systems  Center  (TSC)  to  evaluate  specific 
projects. 

The  objective  of  these  guidelines  is  to  foster 
consistency  of  evaluation  philosophy  and  techniques,  and 
comparability  of  results  so  as  to  improve  the  quality  and 
utility  of  information  obtained  from  the  UMTA  demonstration 
program. 

The  various  demonstration  projects  implemented  under 
the  SMD  Program  are  meant  to  serve  as  learning  tools  and/or 
as  models  for  other  locales  throughout  the  country.  In 
order  for  these  projects  to  have  maximum  effectiveness  in 
their  respective  demonstration  capacities,  a consistent, 
carefully  structured  approach  to  project  evaluation  is 
desirable . 

This  document  has  been  prepared  to  provide  a common 
framework  for  developing  and  then  executing  the  evaluation 
of  individual  demonstration  projects.  These  evaluation 
guidelines  are  by  no  means  comprehensive — that  is,  they  do 
not  offer  a suggested  or  preferred  course  of  action  for 
every  conceivable  situation  that  might  arise.  Nor  are  they 
to  be  rigidly  or  blindly  followed,  since  each  demonstration 
and  each  site  will  be  unique  and  will  require  somewhat 
tailor-made  evaluation  procedures. 

It  is  anticipated  that  these  quidelines  will  be 
modified  during  the  course  of  the  SMD  Program  to  reflect 
experience  gained  in  implementing  and  monitoring  the 
evaluations  of  individual  projects.  Although  it  is  not  the 
desire  to  update  these  Guidelines  frequently,  modifications 
resulting  from  field  experience  will  be  made  where 
appropriate  for  enhancement  of  performance  and  evaluation  of 
the  various  projects. 

In  order  to  put  these  guidelines  into  a meaningful 
context.  Chapter  II  provides  background  information  on  the 
UMTA  Service  and  Methods  Demonstration  Program  and  the 
demonstration  evaluation  process.  Chapters  III  and  IV 
present  guidelines  relative  to  planning  and  executing 
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demonstration  evaluations.  Finally,  Chapter  V presents  the 
recommended  content  and  organization  for  each  type  of  report 
to  be  prepared  in  conjunction  with  the  evaluation  process. 
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CHAPTER  II 


BACKGROUND  ON  THE  SERVICE  AND  METHODS  DEMONSTRATION 
PROGRAM  AND  THE  EVALUATION  PROCESS 


A.  DESCRIPTION  OF  THE  SERVICE  AND  METHODS  DEMONSTRATION 

PROGRAM 

The  Service  and  Methods  Demonstration  (SMD)  Program 
being  sponsored  by  UMTA  is  intended  to  develop,  demonstrate, 
and  evaluate  new  techniques  and  methods  for  using  the 
current  generation  of  transit  equipment  in  providing  a 
significantly  improved  quality  and  quantity  of  public 
transportation.  UMTA's  Office  of  Service  and  Methods 
Demonstrations  intends  to  sponsor  a number  of  demonstrations 
in  various  cities  throughout  the  country.  Results  of  the 
program  will  be  disseminated  in  coordination  with  the  UMTA 
Office  of  Transit  Management. 

Initially,  the  SMD  Program  is  emphasizing  five 
ob jecti ves: 

(1)  Reducing  travel  time  for  transit  users 

(2)  Increasing  transit  coverage 

(3)  Improving  the  reliability  of  transit  service 

(4)  Increasing  transit  vehicle  productivity 

(5)  Improving  service  for  the  transit  dependent.* 

The  demonstrations  undertaken  to  promote  one  or  more  of 
these  five  objectives  can  be  categorized  as  exemplary  or 
experimental. 

Exemplary  demonstrations  are  designed  to  encourage  the 
implementation  of  innovative  transit  services  and  methods  on 
a nationwide  basis  by  increasing  public  exposure  to  proven 
techniques.  The  emphasis  in  this  type  of  demonstration  is 
to  synthesize  and  apply  techniques  which  have  already  been 
developed  and  demonstrated  to  a reasonable  degree  on  an 
experimental  basis. 


♦Transit  dependent  persons  are  those  who  because  of  age, 
income  or  physical/mental  incapability  must  rely  on  public 
transportation,  i.e.,  do  not  have  use  of  automobiles  except 
as  passengers.  Groups  included  as  transit  dependents  are 
the  elderly,  handicapped,  youth  and  poor  (unemployed  as 
well  as  non-members  of  the  labor  force) . 
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In  contrast,  the  experimental  demonstrations  are 
investigative  in  nature,  serving  to  develop  innovative 
services  and  methods  to  the  stage  where  they  are  appropriate 
for  an  exemplary  demonstration.  As  a research  base  for 
exemplary  demonstrations,  the  experimental  demonstrations 
focus  upon  addressing  specific  questions  or  providing 
particular  services,  rather  than  adopting  the  more 
comprehensive  service  philosophy  of  the  exemplary 
demonstrations.  Studies  may  investigate  potential 
experiments  and  isolate  the  factors  to  be  examined  in 
limited  scale  demonstrations.  These  experimental 
demonstrations  then  become  the  developmental  phase  of  new 
concepts. 

In  order  for  the  SMD  Program  to  encourage  significant 
transit  innovations  by  many  urban  areas,  the  techniques 
demonstrated  and  the  results  obtained  must  be  well 
documented  and  widely  distributed.  It  is  important  not  only 
that  the  demonstrations  be  structured  so  as  to  be 
transferrable  but  also  that  the  information  be  disseminated 
in  such  a way  that  the  appropriate  officials  in  those  urban 
areas  which  might  benefit  from  the  application  of  these 
techniques  are  made  aware  of  their  potential.  Accordingly, 
a significant  element  of  the  SMD  Program  is  the  information 
dissemination  function.  UMTA  is  preparing  an  integrated 
plan  for  dissemination  of  the  experience  and  knowledge 
gained  from  the  demonstrations  to  a wide  variety  of  target 
groups. 

The  exact  number,  general  content,  and  location  of  the 
SMD  projects  are  yet  to  be  determined.  For  each  fiscal  year 
program,  a series  of  primary  demonstration  objectives  will 
be  selected,  and  a group  of  demonstration  proposals 
corresponding  to  each  objective,  and  in  keeping  with  total 
budgetary  constraints,  will  be  developed.  Then,  following 
an  investigation,  analysis,  and  negotiation  process 
involving  UMTA,  TSC,  and  candidate  sites,  a final  set  of 
demonstration  projects  and  respective  sites  will  be  agreed 
upon.  Once  final  negotiation  and  transfer  of  funds  between 
UMTA  and  a particular  local  grant  recipient  are  completed, 
the  demonstration  can  be  implemented  and  evaluated. 

As  part  of  its  responsibility  to  evaluate  the 
demonstrations  implemented  under  the  SMD  Program,  TSC  has 
engaged  contractor  support  to  participate  in  all  phases  of 
the  evaluation  process.  Since  it  is  anticipated  that 
several  demonstrations,  which  will  be  geographically 
dispersed,  will  be  in  progress  simultaneously,  a team  of 
contractors  has  been  engaged  for  this  purpose. 
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Exhibit  1 shows  the  interaction  among  UMTA,  TSC,  the 
local  grant  recipient  for  a particular  SMD  project,  and  the 
evaluation  contractor  assigned  to  that  project. 


EXHIBIT  1.  DEMONSTRATION  ORGANIZATIONAL  RELATIONSHIP 


UMTA's  Office  of  Service  and  Methods  Demonstrations  is 
responsible  for  overseeing  and  guiding  all  aspects  of  the 
demonstration,  including  planning,  site  selection, 
negotiations  with  the  site,  implementation,  and  evaluation. 
The  local  grant  recipient  is  responsible  for  planning  and 
implementing  the  operational  phase  of  the  demonstration 
project,  as  well  as  performing  most  of  the  data  collection. 
TSC  assists  UMTA  in  the  activities  for  which  UMTA  is 
responsible,  and  monitors  the  efforts  of  the  evaluation 
contractor.  Both  TSC  and  the  evaluation  contractor 
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interface  with  the  grant  recipient  (or  the  implementing 
agency,  if  different  from  the  grant  recipient).  While  being 
directly  responsible  to  TSC  for  its  activities,  the 
evaluation  contractor  will  maintain  an  informal  association 
and  relationship  with  both  the  grant  recipient  and  the 
cognizant  UMTA  Project  Manager.  It  is  essential  to 
recognize  that  the  evaluation  contractor  performs  his 
functions  as  an  agent  of  TSC  and,  hence,  must  maintain  this 
channel  of  communication  and  reporting. 


B.  OVERVIEW  OF  THE  EVALUATION  PROCESS 

The  evaluation  process  can  be  conceptually  thought  of 
as  a link  between  the  demonstration  and  information 
dissemination  portions  of  the  SMD  Program.  That  is,  it 
serves  as  a bridge  between  the  operation  of  a demonstration 
project  at  a particular  site  and  the  understanding  of  its 
actual  performance  at  that  site  as  well  as  its  potential 
effectiveness  in  other  locales.  Whether  the  project  is 
exemplary  or  experimental,  the  quality  of  the  evaluation 
process  directly  influences  the  accuracy  and  perceptiveness 
of  the  demonstration  assessment  and  ultimately  affects  the 
applicability  and  transferability  of  project  findings. 

Exhibit  2 is  a flow  diagram  representing  the  evaluation 
process  for  an  SMD  project.  The  diagram  is  divided  into 
four  major  sections:  the  evaluation  frame  of  reference, 

evaluation  planning,  evaluation  implementation,  and 
potential  evaluation  spin-offs.  (The  specific 
organizational  responsibilities  associated  with  the  various 
aspects  of  each  SMD  project  are  given  in  Exhibit  4 later  in 
this  chapter.)  The  first  and  fourth  sections  can  be  thought 
of,  respectively,  as  input  to  and  output  from  the  active 
phase  of  the  evaluation  process,  planning  and 
implementation.  A discussion  of  each  of  the  four  sections 
follows. 


1 . Eva 1 ua ti on  Frame  of  Reference 

The  evaluation  frame  of  reference  consists  of  four 
"given"  elements  for  an  evaluation:  the  demonstration 
project,  SMD  Program  objectives,  other  relevant  project 
objectives/issues,  and  the  demonstration  site. 
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Typically,  an  SMD  project  consists  of  some  combination 
of  service-related  and/or  methods- related  transit 
innovations  which  are  introduced  simultaneously  or  in 
sequence.  Exhibit  3 presents  examples  of  innovations  which 
might  be  included  in  SMD  projects.  This  is  not  meant  to  be 
an  exhaustive  list,  but  rather  serves  as  an  indication  of 
the  range  of  services  and  techniques  which  might  be  applied. 

The  project  is  classified  as  exemplary  or  experimental 
depending  on  how  innovative  the  various  components  are  and 
on  the  envisioned  function  of  the  project — showcase  for 
other  cities  to  emulate  vs.  testing  ground  for  unproven 
techniques. 

Each  SMD  project  (or  individual  elements  thereof)  is 
intended  to  serve  one  or  more  SMD  program  objectives.  The 
five  initial  program  objectives  deal  with  user-related 
improvements  (travel  time,  coverage,  reliability),  operator- 
related  improvements  (productivity) , and  service 
improvements  for  a special  market  segment,  the  transit 
dependent. 

For  any  given  project,  there  will  be  objectives,  over 
and  above  the  addressed  SMD  Program  objectives,  which  should 
be  considered  in  the  evaluation.  These  might  be  objectives 
which  the  locality  is  striving  to  attain  (for  example,  a 
certain  percentage  reduction  in  vehicle  travel  or  downtown 
parking  requirements)  or  objectives  which,  while  not 
specifically  transit-oriented,  certainly  have  both  a local 
and  national  bearing  (examples  of  which  are  pollution 
reduction  and  energy  conservation) . 

The  relevant  SMD,  local,  and  national  objectives 
describe  the  desired  or  anticipated  impact  of  the  project  on 
the  transit  user,  operator,  and  general  community.  In 
addition  to  these  objectives,  there  will  generally  be  a set 
of  issues,  or  research  questions,  associated  with  a 
demonstration  project.  These  might  concern  operational 
aspects  of  the  project  innovations  (e.g.,  safety,  ease  of 
implementation,  public  acceptance)  or  might  be  of  an 
exploratory  nature  (e.g.,  assessing  passenger  sensitivity  to 
transfers).  It  is  anticipated  that  experimental 
demonstrations  will  be  more  issue-oriented  than  exemplary 
demonstrations,  owing  to  the  many  more  unknowns  regarding 
operational  feasibility  and  characteristics  of  the 
demonstrated  services  or  methods. 
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EXHIBIT  3 


TYPICAL  SERVICE  AND  METHODS  DEMONSTRATION 
PROGRAM  PROJECT  INNOVATIONS 


CATEGORY  OF  INNOVATION 

EXAMPLES 

Service  Additions/Expansions 

Fixed  route,  demand  responsive, 
subscription  buses  and  vans, 
para-transit,  shared  taxi, 
multi-user  auto  system,  jitney, 
light/heavy  rail  service, 
shared  ride  auto. 

Service  Improvements 

Increased  service  frequency, 
improved  coordination  at  transfer 
points,  park-and-ride  lots. 

New  Types  of  Buses 

Small  buses.  Transbus,  double  deck 
bus,  articulated  bus,  special  bus 
for  wheelchair-confined  passengers, 
electric  buses. 

Operational  Improvements 

Manual/automatic  vehicle  moni- 
toring, surveillance,  command  and 
control  systems. 

Preferential  Treatment  of 
Buses /Car pools 

Exclusive/reserved  lanes  (with-flow 
or  contraflow),  ramp  metering, 
priority  traffic  signalization 
systems,  bus-only  streets,  auto 
restricted  zones. 

Pricing/Fare  Collection 
Policies 

No  fare,  off-peak  fare  reductions, 
special  fares  for  the  elderly, 
congestion  pricing,  credit  card 
billing,  prepaid  passes,  tokens. 

Institutional  Changes 

Parking  restrictions,  auto  restricted 
zones,  staggered  work  hours,  transit/ 
carpool  marketing  programs. 
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The  demonstration  site  can  consist  of  anything  from  a 
portion  of  a city  to  a group  of  cities  or  towns,  and  can  be 
at  any  point  along  the  population  and  density  spectrum-  An 
understanding  of  the  unigue  demographic,  economic, 
geographic,  and  transportation  characteristics  of  the  site, 
as  well  as  prevailing  attitudes  toward  transportation,  is  a 
useful  and  necessary  adjunct  to  knowledge  about  the 
demonstration  project  elements  and  objectives. 

The  planned  innovations,  project  objectives  and  issues, 
and  site  characteristics  will  generally  be  available  from 
the  demonstration  grant  application  submitted  to  UMTA  by  the 
site  prior  to  approval  of  the  project.  Depending  on  the 
timing  of  the  contractor's  initial  involvement  in  the 
project,  a more  detailed  description  of  the  project  may  be 
available  in  the  form  of  a Project  Implementation  Plan. 
Further  background  on  the  demonstration  — e.g.,  genesis  of 
the  project  concept,  recent  history  of  transit/para-transit 
developments  at  the  site  — can  be  obtained  through 
discussions  with  the  UMTA  Project  Manager,  TSC  staff,  and 
the  grantee. 

2 . Evaluation  Planning 

The  evaluation  planning  phase  of  the  demonstration 
evaluation  process  is  the  period  during  which  the  contractor 
interacts  with  UMTA,  TSC,  and  various  agencies  at  the  local 
level  to  transform  the  evaluation  frame  of  reference  into  a 
detailed,  structured  program  for  conducting  the  evaluation. 
This  phase  sets  the  stage  for  the  entire  evaluation  effort 
and  in  addition  provides  an  opportunity  to  reassess  and,  if 
necessary,  restructure  the  planned  demonstration  project. 

The  planning  phase  begins  with  TSC's  preparation  of  an 
Evaluation  Framework  for  the  particular  project,  which 
describes : 


(1)  Pertinent  information  on  the  project  and  site  (in 
particular,  an  indication  of  what  features  of  the 
demonstration  are  unique  and  merit  emphasis  in  the 
evaluation)  . 

(2)  SMD  Program  objectives  addressed  by  the 
demonstration. 

(3)  Relevant  local  and/or  national  objectives 
addressed  (and  the  relative  emphasis  to  be  placed 
on  these  objectives  vs.  SMD  objectives) . 
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(4)  Key  issues  to  be  addressed. 

(5)  Recommended  scope  and  focus  of  the  evaluation.. 


It  should  be  noted  that  the  contents  of  each  Evaluation 
Framework  will  vary  from  demonstration  to  demonstration, 
depending  on  the  nature  and  timing  of  the  project. 

The  Evaluation  Framework  becomes  the  basis  for  the  more 
detailed  Evaluation  Plan*  which  is  developed  by  the 
contractor.  While  TSC  will  provide  a general  evaluation 
approach  and  may  suggest  some  measures  to  be  collected,  it 
is  really  left  to  the  contractor  to  refine  and  elaborate  on 
TSC's  suggested  approach  by  developing  specific  procedures 
for  collecting  and  analyzing  data  relative  to  project 
objectives,  issues,  and  the  site. 

In  developing  the  Evaluation  Plan,  the  contractor  is 
encouraged  to  propose  changes  to  the  approach  recommended  by 
TSC  in  its  Evaluation  Framework,  particularly  if  the 
proposed  modifications  have  significant  potential  to  improve 
the  objectivity,  accuracy,  completeness,  and/or  efficiency 
of  the  project  evaluation  effort  or  to  enhance  the 
transferrability  of  project  findings.  In  addition,  total 
evaluation  costs  relative  to  potential  findings  must  be 
borne  in  mind  at  all  times.  Throughout  the  process  of 
developing  the  Evaluation  Plan,  the  contractor  is  urged  to 
keep  in  close  contact  with  the  local  agency  responsible  for 
implementing  and  operating  the  demonstration  and  performing 
data  collection.  This  continuing  liaison  with  the  UMTA 
grant  recipient  will  ensure  that  the  proposed  methods  of 
data  collection  are  consistent  with  the  resources  available 
at  the  local  level,  with  the  demonstration  implementation 
plan  developed  by  the  site,  and  with  reasonable  costs  for 
the  evaluation  contractor  efforts. 

As  is  apparent  from  the  preceding  discussion,  the 
evaluation  planning  phase  entails  substantial  and  continued 
interaction  among  all  parties  involved  in  the  demonstration 
project.  Ideally,  planning  of  the  evaluation  effort  should 
be  coordinated  with,  and  take  place  concurrently  with, 
planning  of  the  project  itself.  This  coordination  between 


♦Chapter  III  presents  guidelines  relative  to 
planning  phase.  The  recommended  content  and 
the  Evaluation  Plan  are  presented  in  Chapter 


the  evaluation 
organization  of 

V. 


2-9 


the  implementation/operation  and  evaluation  planning  cycles 
permits  optimum  flexibility  in  the  conduct  of  the  overall 
demonstration  project.  If  necessary,  operational  aspects  of 
the  demonstration  can  be  planned  to  conform  to  requirements 
of  the  evaluation,  rather  than  the  evaluation  having  to  be 
integrated  into  a pre-existing,  rigid  operational  structure. 
The  concurrence  of  the  two  planning  cycles  ensures  that  the 
Evaluation  Plan  is  completed  prior  to  the  implementation  of 
the  project.  Early  development  of  the  Plan,  in  turn,  allows 
the  necessary  lead  time  for  "before"  data  collection  — that 
is,  observations  of  transit  system  performance  prior  to  the 
introduction  of  the  transit  innovation  (s)  as  well  as 
possible  information  on  community  awareness  and  attitudes 
prior  to  project  implementation.  Throughout  this  phase  of 
the  project,  it  is  critical  to  recognize  that  the  UMTA 
Project  Manager  has  the  final  say  with  respect  to  any 
modifications  in  the  project  itself  as  well  as  the  elements 
of  the  evaluation  plan. 


3 . Evaluation  Implementation 

The  evaluation  implementation  phase  is  the  period 
during  which  the  approved  Evaluation  Plan  is  executed. 
Activities  during  this  phase  include  collection/analysis  of 
data  relative  to  project  objectives  and  issues, 
collection/ analysis  of  data  on  site  characteristics, 
compilation  of  a chronology  describing  the  implementation 
and  operation  of  the  demonstration,  and  recording  of 
exogenous  factors  which  might  influence  demonstration 
findings.  Contractor  functions  during  this  phase  include 
monitoring  of  the  data  collection  process  (generally  to  be 
performed  by  the  local  grant  recipient) , any  data  collection 
not  performed  by  the  grant  recipient,  data  reduction  and 
analysis,  subjective  analysis  of  information  relative  to 
project  issues,  and  synthesis  of  project  findings  into  one 
or  more  Interim  Evaluation  Reports  and  a Final  Evaluation 
Report. * 


♦Chapter  IV  presents  guidelines  relative  to  the  evaluation 
implementation  phase.  Chapter  V gives  the  recommended 
content  and  organization  of  the  various  contractor  reports 
prepared  during  this  phase,  including  the  Monthly  Evaluation 
Progress  Report,  the  Annual  Project  Status  Summary,  the 
Interim  Evaluation  Report,  and  the  Final  Evaluation  Report. 
In  addition.  Chapter  V describes  the  content  of  the 
grantee's  Quarterly  Project  Progress  Report  to  UMTA,  which 
can  serve  as  useful  input  to  the  contractor's  work. 
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This  phase  not  only  generates  information  on  which  the 
final  assessment  of  the  project  is  based  but  also  provides 
feedback  information  relative  to  ongoing  project  operations. 
The  ongoing  evaluation  activities,  while  adding  to  the 
cumulative  body  of  quantitative  and  qualitative  information 
regarding  the  project's  impacts,  provide  interim  indications 
of  how  the  various  innovations  are  functioning  and  the 
preliminarv  effects  of  these  innovations  on  such  areas  as 
patronage,  travel  time,  and  rider  attitudes.  These  interim 
findings  serve  as  useful  input  to  the  local  agency 
responsible  for  implementing  and  operating  the 
demonstration,  by  suggesting  the  need  for  operational 
modifications.  For  example,  in  a project  involving  a 
secruence  of  planned  innovations  (say,  experimentation  with 
fares  or  service  freauencies) , the  finding  that  ridership 
levels  have  stabilized  sooner  than  anticipated  would  make  it 
possible  to  expand  the  list  of  experimental  options  being 
implemented. 

During  this  phase,  modifications  may  be  made  to  the 
evaluation  procedures  originallv  specified  in  the  Evaluation 
Plan.  For  instance,  examination  of  interim  findings  may 
reveal  certain  gaps  or  redundancies  in  the  originally 
planned  data  collection  program.  Still  other  reasons  for 
modifying  the  evaluation  procedure  might  be  operational 
changes  in  the  project,  unanticipated  developments  in  the 
site,  or  discovery  of  an  improved  evaluation  procedure. 
Procedural  steps  to  accomplish  this  necessary  update  for  the 
^valuation  Plan  appear  in  Chapter  V. 

The  culmination  of  the  evaluation  implementation  phase 
is  the  Pinal  Evaluation  Report,  which  presents  the  following 
types  of  findings: 

(1)  Assessment  of  the  project  in  terms  of  its 
attainment  of  relevant  SMD  Program  objectives  and 
other  (local  and/or  national)  project  objectives. 

(2)  Insight  into  project  issues  associated  with 
operational  feasibility  and  characteristics  of  the 
demonstrated  innovations. 

(3)  Assessment  of  the  influence  of  site-specific 
characteristics  and  exogenous  factors  on  the 
outcome  of  the  demonstration. 

(4)  Lessons  learned,  based  on  practical  experience, 
relative  to  the  implementation  and  operation  of 
the  demonstrated  innovations  (possibly  to  include 
recommendations  for  project  modifications  in  the 
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demonstration  site  or  for  future  applications  in 
other  locales). 

(5)  Assessment  of  the  evaluation  procedures  employed 
in  terms  of  cost,  accuracy,  etc. 

In  essence,  the  Final  Report  presents  an  assessment  of  the 
impact  of  the  innovations  at  the  site  and  provides  guidance 
for  their  transfer  to  other  locales. 

The  Final  Evaluation  Report  relies  on  both  narrative 
and  graphic  exposition,  with  detailed  quantitati ve  data  and 
documentation  of  procedures  relegated  to  technical 
appendices.  Since  the  report  is  intended  for  a variety  of 
audiences  --  including  transportation  planners,  transit 
operators,  and  federal,  state,  and  local  officials  — it 
contains  an  Executive  Summary  which  highlights  the  salient 
project  findings. 


4 . Potential  Evaluation  Spin-offs 

It  is  anticipated  that  each  demonstration  will  give 
rise  to  potential  implementation  and  analytical  spinoffs. 

The  Final  Evaluation  Report,  while  essentially  documenting 
the  history  and  effects  of  a single  case  study  project,  also 
serves  the  broader  function  of  increasing  the  understanding 
of  and  stimulating  the  application  of  the  demonstrated 
services  and/or  methods  in  other  localities.  Information 
presented  in  the  report  provides  a versatile  basis  for 
comparing  the  effects  of  the  particular  demonstration  with 
those  of  other  similar  projects,  suggesting  modifications  to 
the  demonstrated  services/methods  for  future  application, 
and  predicting  the  effectiveness  and  applicability  of  the 
demonstrated  services/  methods  in  other  cities.  Moreover, 
the  report’s  assessment  of  project  evaluation  procedures  can 
serve  as  a stimulus  for  improving  the  state-of-the-art  of 
evaluation  techniques.  Since  these  broader  functions  of  the 
Final  Evaluation  Report  generally  materialize  after  the 
demonstration  period  and  are  not  within  the  purview  of  the 
evaluation  contractor  assigned  to  a particular  project,  they 
are  shown  in  Exhibit  2 as  potential  evaluation  spin-offs. 
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c. 


COORDINATION  OF  SMD  EVALUATIONS 


Exhibit  4 summarizes  the  various  activities  involved  in 
planning,  implementing,  and  evaluating  an  SMD  project  and 
indicates  the  allocation  of  responsibility  for  these 
activities.  The  sequence  of  activities  ranges  from  overall 
SMD  Program  definition,  to  the  operation  and  evaluation  of 
an  individual  demonstration,  to  the  spin-off  uses  of  the 
project.  It  can  be  seen  that  the  entire  stream  of 
activites,  especially  those  comprising  the  evaluation 
process,  involves  extensive  interaction  among  UMTA,  the 
grant  recipient,  TSC,  and  the  evaluation  contractor. 
Moreover,  it  should  be  noted  that  the  activities  shown  do 
not  always  occur  in  a fixed  sequence.  Time  constraints  may 
require  that  some  of  the  steps  be  performed  in  parallel,  and 
there  will  generally  be  considerable  interaction  and 
feedback  between  the  project  planning  and  evaluation 
planning  phases.  Evaluation  spin-offs,  while  arising  out  of 
individual  demonstrations,  will  result  in  activities  which 
extend  beyond  the  UMTA,  TSC,  grantee,  and  evaluator 
organizations. 

The  diversity  of  activities  and  generally  long  (two  to 
three  year)  time  frame  for  an  individual  project  necessitate 
close  and  continual  coordination  among  the  groups  involved. 
However,  equally  as  important  as  coordination  within  a 
particular  project  is  coordination  across  demonstration 
projects,  so  as  to  maximize  the  effectiveness  of  the  SMD 
Program  in  encouraging  the  demonstration  and  application  of 
new  services  and  methods.  This  coordination  across  projects 
is  especially  important  with  respect  to  the  evaluation 
process.  Given  the  multiplicity  of  sites,  demonstrations, 
and  participating  organizations  within  the  SMD  Program, 
there  is  a strong  need  for  coordination  of  the  demonstration 
evaluation  process  so  as  to  achieve  consistency  in  the 
planning,  implementation,  and  output  of  individual  project 
evaluations. 

With  respect  to  the  conduct  of  the  evaluations,  such 
coordination  will  ensure  that:  (D  the  scope  of  each 

evaluation  effort  is  consistent  with  the  importance  of  that 
particular  demonstration  project  relative  to  other  SMD 
projects;  and  (2)  the  technical  approaches  used  to  evaluate 
projects  are  consistent  with  the  current  state-of-the-art  of 
evaluation  techniques. 

With  respect  to  evaluation  output,  such  coordination 
will  ensure  that  the  Final  Evaluation  Reports  associated 
with  individual  projects  are  consistent  in  terms  of  content, 
format,  perspective,  and  level  of  detail,  thereby 
facilitating  their  synthesis  into  an  Annual  Service  and 
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EXHIBIT  4 

SMD  PROJECT  PLANNING/IMPLEMENTATION/EVALUATION: 
SEQUENCE  OF  AND  RESPONSIBILITY  FOR  ACTIVITIES 


Responsibility  for  Activities 

Category 
of  Activity 

Activity 

UMTA 

Grant 

Recipient 

TSC 

Evaluation 

Contractor 

SMD  Program 

Establishment  of  SMD  Program 
objectives 

P 

S 

SMD  Program 

Identification  of  candidate 
si tes/demonst rat ions 

P 

S 

Demo-operation 

UMTA/Site  negotiations 

P 

P 

Demo-operation 

Development  of  final  demon- 
stration description  and  im- 
plementation plan 

R 

P 

R 

Demo-evaluation 

Development  of  Evaluation 
Framework 

R 

R 

P 

Demo-evaluation 

Development  of  Evaluation  Plan 

R 

R 

M 

P 

Demo-evaluation 

"Before"  data  collection 

P 

M 

Demo-operation 

Demonstration  implementation 

M 

P 

M 

M 

Demo-evaluation 

Data  collection 

P 

M 

Demo-eval  uation 

Data  reduction  and  analysis 

M 

P 

Demo-oper/eval 

Preparation  of  Quarterly  Project 
Progress  Reports 

R 

P 

Demo-evaluation 

Preparation  of  Monthly  Evaluation 
Progress  Reports 

R 

R 

P 

Demo-eval  uat'1'  on 

Preparation  of  Annual  Project  Status 
Summaries 

R 

R 

R 

P 

Demo-eval  uation 

Preparation  of  Interim  Evaluation 
Report(s) 

R 

R 

R 

P 

Demo-evaluation 

Preparation  of  Final  Evaluation 
Report 

R 

R 

R 

P 

Evaluation 

Spin-off 

Inter-project  comparisons 

R 

P 

Evaluation 

Spin-off 

Post-demonstration  project 
modifications  at  site 

R 

P 

Eval uation 
Spin-off 

Improvement  in  evaluation 
methodology 

R 

R 

P 

Evaluation 

Spin-off 

Application  of  project  findings  to 
innovations  in  other  sites 

Other  sites 

Evaluation 

Spin-off 

Use  of  project  data  base  in  simula- 
tion models 

Transportation 

pi anners 

P = primary  role 
M = monitoring  role 
R = review  function 
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Methods  Demonstration  Program  Report.  This  consistency  in 
output  will,  in  addition,  enhance  the  spin-off  potential  of 
the  evaluations.  The  achievement  of  a basic  data  set  of 
uniform  quality  across  demonstrations  will  make  possible 
interproject  comparisons  in  terms  of  system  user 
cha rateristics , site  characteristics,  and  system  performance 
and  financial  measures.  These  types  of  comparisons  will  be 
especially  significant  in  the  case  of  multiple  applications 
of  a particular  service  or  method  in  several  locations,  or 
in  the  case  of  demonstrations  involving  alternative  services 
or  methods  directed  towards  a particular  SMD  Program 
ob jecti ve . 

The  coordination  of  the  individual  evaluation  efforts 
will  be  achieved  through  TSC's  active  and  continual 
participation  in  the  program,  with  functions  ranging  from 
initial  planning  of  each  project  evaluation  effort  (the 
Evaluation  Framework) , to  monitoring  of  the  contractor  team, 
and  finally  to  synthesis  of  individual  demonstrations, 
evaluation  reports  and  results  into  an  annual  SMD  report. 
This  guidelines  document  constitutes  the  first  stage  of 
TSC's  evaluation  coordination  function,  in  that  it  describes 
general  procedures  to  be  followed  by  each  contractor  in 
performing  the  various  evaluation  tasks  specified  in  the 
contract. 
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CHAPTER  III 


GUIDELINES  FOR  PLANNING  EVALUATION  ACTIVITIES 


This  chapter  presents  specific  guidelines  for  planning 
the  evaluation  activities  associated  with  a particular  SMD 
project.  As  was  mentioned  in  Chapter  II,  the  evaluation 
planning  phase  of  the  demonstration  evaluation  process  is 
that  period  during  which  the  contractor  prepares  a detailed 
Evaluation  Plan  based  on  TSC's  Evaluation  Framework.  The 
Evaluation  Plan  contains,  among  other  things,  a listing  of 
relevant  guantitative  and  qualitative  measures  related  to 
various  SMD,  local,  and  national  objectives  and  relevant 
issues,  associated  data  collection  and  analysis  procedures, 
and  site  specific  data  requirements  and  sources  (both  one- 
time and  recurring) . As  such,  this  plan  constitutes  a 
structured,  time-phased  program  for  subsequently  conducting 
the  evaluation. 

The  chapter  is  organized  into  three  sections, 
corresponding  to  the  basic  decision-making  elements  shown  in 
Exhibit  2: 


• Determination  of  site  data  requirements  and 
sources 

• Determination  of  measures  and  collection/deriva- 
tion techniques  required  to  address  SMD  Program 
objectives  and  other  relevant  objectives/issues 

• Planning  considerations  relative  to  data 
collection  and  analysis. 

The  organization  of  the  chapter  is  not  meant  to  imply  a 
highly  ordered  time-sequencing  of  activities,  since  the 
evaluation  planning  phase  is  in  fact  highly  iterative  and 
dynamic.  Moreover,  it  is  important  to  realize  that  these 
guidelines  comprise  a basic  set  of  ground  rules  for  planning 
evaluations.  The  evaluation  contractor  will,  in  all 
probability,  need  to  depart  from  these  guidelines  during  the 
actual  planning  phase,  so  as  to  conform  to  the  unique 
conditions  surrounding  a given  demonstration. 

The  contractor  should  recognize  his  responsibility  in 
working  with  the  grantee  and  TSC  to  assure  that  an  objective 
assessment  of  the  project  is  achieved.  One  or  more  site 
visits  during  the  evaluation  planning  phase  is  desirable  to 
establish  working  relationships  and  channels  of 
communication  among  the  various  involved  organizations  and 
to  uncover  any  constraints  which  may  have  a significant 
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bearing  on  the  development  of  the  Evaluation  Plan.  During 
this  planning  effort,  clarification  must  be  made  regarding 
responsibilities  for  performing  and/or  overseeing  various 
activities.  The  Evaluation  Plan  should  indicate  the  finally 
agreed  upon  allocation  of  responsibility  between  the 
contractor  and  local  organizations. 


A.  DETERMINATION  OF  SITE  DATA  REQUIREMENTS  AND  SOURCES 

The  purpose  of  the  site  data  is  to  provide  an  in-depth 
understanding  of  those  characteristics  of  the  site  which 
might  in  some  way  influence  the  outcome  of  the  project  or 
the  interpretation  of  project  results.  Obviously,  the  SMD 
project  will  not  be  implemented  in  a static  environment,  but 
rather  it  will  affect  the  surrounding  area.  Thus  an 
examination  of  certain  site  characteristics  is  necessary  in 
order  to  assess  fully  and  accurately  the  impact  of  the 
project. 

An  additional  function  of  site  data  is  to  enhance  the 
comparability  and  transferability  of  SMD  project  findings. 
Specifically,  if  conclusions  drawn  from  one  project  are  to 
be  compared  with  findings  of  other  similar  projects  or 
"transferred"  to  other  potential  sites,  there  must  exist  an 
objective  approach  for  such  a comparison  or  transfer.  This 
requires  the  identification  of  a set  of  site-specific 
measures  which  permit  one  to  classify  sites  in  terms  of 
meaningful  similarities,  or  to  identify  significant  areas  in 
which  sites  differ,  such  as  socio-economic  characteristics. 
Institutional  groups  which  must  decide  how  to  allocate  their 
available  funds  for  transportation  system  improvement  need 
to  be  able  to  compare  their  site  environment  with  that  of 
demonstration  sites. 

Examples  of  site  data  which  might  be  relevant  are 
demographic  and  land  use  characteristics,  transportation 
facilities,  and  vehicle  travel  characteristics,  both  intra- 
and  inter-urban.  In  addition,  information  on  the 
political/institutional  climate  of  the  area  and  prevailing 
attitudes  toward  transportation-related  issues  might  be 
helpful  in  anticipating  or  understanding  any  problems 
regarding  implementation  and  evaluation  of  the  project. 

A review  of  recent  demonstration  projects  indicates  an 
inconsistency  in  both  amounts  of  and  details  for  reported 
site-specific  data.  To  some  extent  this  inconsistency 
reflects  a lack  of  standardized  site  data  requirements,  but 
more  significantly  it  reflects  deficiencies  in  knowledge 
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regarding  the  interplay  between  site  characteristics  and 
demonstration  results.  In  an  attempt  to  shed  further  light 
on  the  subject,  a basic  set  of  data  requirements  has  been 
developed  for  use  in  SMD  projects  (see  Exhibit  5) . 

Contractors  are  encouraged  to  propose  additions  or 
deletions  to  this  list,  in  the  context  of  particular 
projects,  if  it  is  felt  that  the  nature  and  scope  of  the 
project  call  for  a wider  or  narrower  set  of  site 
descriptors.  If  the  project  is  relatively  self-contained 
(e.g.,  a demonstration  of  a new  vehicle  concept),  then  site 
data  collection  need  not  encompass  broad  site  descriptors 
such  as  demographic  and  land  use  characteristics;  on  the 
other  hand,  a far-ranging  project  (e.g.,  introduction  of  a 
coordinated  feeder/line-haul/distribution  service)  probably 
calls  for  the  complete  gamut  of  site  data  so  as  to 
understand  user/non-user  characteristics,  modal  shift,  and 
integration  of  the  new  service  into  the  existing 
transportation  infrastructure.  Contractors  are  also 
encouraged  to  propose  permanent  additions,  deletions,  or 
changes  to  this  minimum  list  based  on  their  cumulative 
experience  in  conducting  SMD  evaluations. 

Aside  from  the  site  data  requirements  in  Exhibit  5,  it 
may  be  desirable  to  collect  a standardized  set  of 
attitudinal  measures  to  obtain  a profile  of  the  community. 
Examples  would  be  general  opinions  regarding  the  role  of 
government,  environmental  issues,  adequacy  of  transportation 
facilities,  and  desirability  of  travel  by  alternative  modes. 
Since  the  value  of  this  type  of  data  for  demonstration 
evaluation  and  transferability  purposes  has  not  yet  been 
fully  explored,  community  profile  data  will  be  collected 
only  in  selected  demonstrations  (to  be  identified  by  TSC  in 
the  Evaluation  Framework)  . Appendix  A contains  sample 
questionnaires  which  might  be  used  to  obtain  such  data.  As 
experience  is  gained  in  this  area,  a standardized  approach 
to  developing  an  attitudinal  profile  of  the  demonstration 
site  may  be  formally  incorporated  into  these  guidelines. 

It  is  anticipated  that  the  data  set  and  descriptive 
information  shown  in  Exhibit  5 will  be  available  from 
secondary  sources  or  from  the  grantee  and  will  not  involve 
specialized  data  collection  activities  (an  exception  being 
attitudinal  profile  data,  which  will  entail  surveys) . 

Exhibit  6 indicates  typical  sources  for  various  categories 
of  site-specific  data. 

Once  the  contractor  has  determined  the  type  of  site 
data  to  be  assembled  and  the  appropriate  sources,  two 
decisions  remain:  (1)  the  geographic  scope  of  the  area  for 
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EXHIBIT  5.  BASIC  SITE  DATA  REQUIREMENTS  FOR  SMD  PROJECTS 
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EXHIBIT  6.  TYPICAL  SOURCES  FOR  SITE  DATA* 
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which  data  is  to  be  assembled,  and  (2)  the  time  period (s) 
for  which  data  is  to  be  assembled. 

Regarding  the  geographic  scope,  it  was  indicated  above 
that  a basic  data  set  should  be  assembled  for  the  SMD 
service  area.*  In  some  cases,  data  conforming  exactly  to 
the  service  area  boundary  may  be  unavailable  or  may  be 
obtained  only  by  aggregation  of  fine-grained  (e.g..  Census 
tract)  data.  If  there  is  available  data  for  an  area 
approximating  the  service  area,  the  contractor  may  choose  to 
use  this  pre-existing  data  base  rather  than  deriving  a 
special  data  base,  provided  that  such  a substitution  will 
not  be  misleading  and  bias  the  evaluation.  On  the  other 
hand,  the  use  of  fine-grained  data  may  be  appropriate  if  the 
service  area  is  large  and  heterogeneous  and  thus  should  be 
divided  into  zones. 

The  time  period (s)  for  which  data  is  to  be  assembled 
depends  on  the  time  period  of  the  demonstration  project  and 
the  rate  at  which  conditions  at  the  site  are  changing.  If 
the  project  spans  a fairly  long  period  it  may  be  desirable 
to  gather  site  data  for  periods  before,  during,  and  after 
the  project.  In  the  case  of  a rapidly  changing  area  or  a 
staged  project,  data  for  even  more  points  in  time  may  be 
necessary.  Moreover,  if  an  historical  perspective  on  the 
site  is  deemed  relevant  to  the  evaluation,  it  may  be 
desirable  to  obtain  1960  as  well  as  1970  Census  figures  or 
recent  trend  data  for  key  variables  such  as  population, 
employment,  and  modal  split.  Since  original  data  collection 
by  the  contractor  is  not  anticipated,  the  number  and  exact 
timing  of  site  data  periods  will  be  constrained  by  the 
collection  cycles  of  existing  sources. 


B.  DETERMINATION  OF  MEASURES  AND  COLLECTION/DERIVATION 

TECHNIQUES  REQUIRED  TO  ADDRESS  SMD  PROGRAM  OBJECTIVES 
AND  OTHER  RELEVANT  PROJECT  OBJECTIVES/ISSUES 

It  was  pointed  out  in  Chapter  II  that  TSC* s Evaluation 
Framework  will  set  forth  a recommended  set  of  SMD  Program 


*A  definition  of  the  SMD  service  area  may  not  be  available 
at  the  outset  of  the  project,  but  rather  will  need  to  be 
developed  during  the  evaluation  implementation  phase  on  the 
basis  of  user  surveys. 
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objectives,  relevant  project  objectives  (of  local  and 
national  significance) , and  project  issues  to  be  examined. 
The  contractor,  in  developing  the  Evaluation  Plan,  is 
responsible  for  reviewing  this  recommended  set  in  the 
context  of  the  grantee's  Project  Implementation  Plan  and  the 
various  national  and  local  perspectives,  and  then  proposing 
appropriate  modifications  to  the  list  of  objectives  and 
i ssues. 

Once  the  set  of  project  objectives  and  issues  has  been 
finalized  (which  involves  obtaining  verbal  concurrence  from 
TSC) , the  contractor  must  associate  with  these  items  a set 
of  germane  measures  and  identify  suitable  techniques  for 
collecting  or  deriving  each  measure.  It  is  important  to 
note  that  certain  issues  may  not  lend  themselves  to  the 
collection  of  either  qualitative  or  quantitative  measures 
but  may  rather  involve  subjective  analysis  of  pertinent 
information. 

The  material  presented  below  is  intended  to  guide  the 
contractor  in  developing  appropriate  measures  and  associated 
collection/derivation  techniques.  It  is  important  to 
recognize  that  this  material  will  undoubtedly  be  modified  as 
information  is  gained  through  the  consistent  application  and 
analysis  of  evaluation  techniques  on  the  SMD  projects. 
Therefore,  because  revisions  to  data  program  requirements  in 
terms  of  basic  data  sets,  collection  and  analysis 
procedures,  and  presentation  techniques  can  be  expected,  the 
fundamental  value  of  this  section  of  the  guidelines  lies  in 
the  manner  in  which  it  structures  the  approach  to  the 
selection  of  measures  and  the  selection  of  techniques  for 
collecting/deriving  them. 

In  preparing  this  material,  considerable  documentation 
was  reviewed  (see  Bibliography) . In  addition,  direct 
observance  and  participation  in  many  previous  and  ongoing 
demonstration  projects  has  permitted  those  preparing  this 
document  to  identify  not  only  a logical  structure  for 
project  evaluation  but  also  to  highlight  problem  areas  of 
which  all  potential  project  evaluators  should  be  aware.  The 
specific  demonstration  projects  which  contributed  the 
greatest  amount  of  insight  were  the  Minneapolis  Urban 
Corridor  Project,  Miami  I-95/N.W.  7th  Avenue  Bus/Car  Pool 
Project,  Seattle  Blue  Streak  Project,  and  Shirley  Highway 
Express-Bus-On- Freeway  Project. 
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1 . Ba sic  Set  of  Measures 

Exhibit  7 presents  a basic  set  of  measures  applicable 
for  evaluating  SMD  demonstrations.  The  exhibit  is  divided 
into  six  sections:  the  first  five  contain  measures 

corresponding  to  the  five  SMD  Program  objectives,  and  the 
last  section  consists  of  measures  related  to  other  project 
objectives  and  issues.  For  each  measure  (listed  in  column 
1),  the  following  information  is  indicated: 


• Type  of  measure  (column  2) 

Quantitative  --  a measure  which  is  expressed 
in  terms  of  counts,  measurements,  dollars,  or 
other  physical  units 

Qualitative  — a measure  which  is  expressed 
in  terms  of  people's  attitudes,  perceptions, 
or  observations 

• Method  of  obtaining  measure  (column  3) 

Collected  — obtained  by  measurement  (vehicle 
travel  time) , counting  (number  of  passen- 
gers) , surveying  (perceived  reliability) , or 
from  records  (daily  revenue) 

Derived  — calculated  from  collected  measures 
either  by  simple  arithmetic  procedures 
(passenger  miles  per  seat  mile)  or  through 
use  of  analytic  models  (reduction  in  air 
pollution  or  fuel  consumption) 

• Possible  collection/derivation  techniques 
(column  4) 

Brief  description  of  alternative  methods  for 
collecting  or  deriving  measure 


In  reviewing  the  basic  set  of  measures  in  Exhibit  7,  it  is 
important  to  note  that  some  of  these  measures  would  be  more 
meaningful  if  stratified  by  time  of  day  (peak  versus  off- 
peak)  , location  (corridor  versus  arterial) , person  time 
segments  (waiting,  access,  transfer,  in-vehicle) , route  type 
(fixed  route  versus  demand  responsive) , and  vehicle  tour 
segments  (in-service,  non-service) . Because  such  a 
classification  of  measures  would  have  needlessly  extended 
the  list  in  Exhibit  7,  the  subject  of  stratification,  or 
categorization,  with  respect  to  specific  data  collection 
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EXHIBIT  7.  A SET  OF  BASIC  MEASURES  AND  COLLECTION/DERIVATION 
TECHNIQUES  FOR  SMD  PROJECTS 
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These  ratios  can  be  used  to  convert  productivity  measures  from 
a total  miles  or  total  hours  basis  to  an  in-service  miles  or  hours  basis. 
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plans  is  discussed  separately  in  Section  C,  Part  2 of  this 
chapter. 

The  entire  set  of  measures  in  Exhibit  7 is  not  to  be 
construed  a^  a minimum  requirement  for  every  SMD  project, 
since  an  evaluation  need  only  encompass  measures 
corresponding  to  the  SMD  Program  objectives  and  other 
project  objectives/issues  addressed  by  the  particular 
demonstration.  Nor  is  the  group  of  measures  corresponding 
to  each  of  the  five  SMD  Program  objectives  to  be  considered 
a minimum  set  for  that  objective.  Rather,  Exhibit  7 should 
be  used  by  the  contractor  as  a checklist  from  which  the  most 
germane  measures  can  be  collected  and  to  which  other 
relevant  measures  can  be  added  as  appropriate. 

It  will  be  noted  that  for  each  of  the  SMD  Program 
objectives,  it  is  possible  to  measure  attainment  of  the 
objective  from  two  vantage  points:  the  actual  and  the 
perceived  attributes  of  the  transit  system  (as  represented 
by  quantitative  and  qualitative  measures,  respectively) . In 
the  case  of  transit  travel  time,  it  might  be  appropriate  to 
measure  actual  changes  in  travel  time  as  well  as  to  assess 
user  perceptions  of  changes  in  travel  time  and  then  to 
compare  the  two.  Similarly,  it  might  be  interesting  to 
obtain  non-user  impressions  of  transit  system  coverage  and 
correlate  these  impressions  with  actual  coverage  measures. 

In  the  case  of  productivity  and  equipment  reliability 
measures,  comparisons  with  transit  operator  perceptions 
might  also  be  appropriate. 

Until  more  is  learned  about  the  interrelationship 
between  actual  measurements  and  attitudinal  data,  it  is  not 
possible  to  set  forth  hard  and  fast  rules  for  when  to 
supplement  quantitative  measures  with  qualitative  measures. 
Clearly,  it  would  be  prohibitively  expensive  to  employ  this 
two-pronged  procedure  for  each  area  of  interest;  on  the 
other  hand,  mere  reliance  on  quantitative  measures  may 
result  in  overlooking  what  is  in  fact  the  major  behavioral 
determinant  --  people's  perceptions  of  the  system.  For  the 
time  being,  the  contractor  must  exercise  sound  judgment  in 
deciding  which  situations  are  unique  and  instructive  enouqh 
to  warrant  a two-pronged  collection  effort.  In  no  case 
should  an  attitudinal  measure  ever  be  used  in  place  of  a 
quantitative  measure,  where  both  are  available. 

The  philosophy  or  rationale  underlying  each  group  of 
measures  in  Exhibit  7 is  discussed  below.  Further 
discussion  of  collection/ derivation  techniques  appears  in 
Part  2 of  this  section. 


3-2& 


TRAVEL  TIME 


The  primary  measure  to  be  collected  under  this 
objective  category  is  travel  time  as  measured  in  minutes. 

It  is  fairly  clear  that  transit  travel  time  can  be  addressed 
in  terms  of  the  passenger  (actual  or  perceived)  and  the 
vehicle  (actual  or  scheduled) . Since  the  private  auto  may 
well  be  a factor  in  both  a transit  user's  total  trip  time 
and  as  an  alternative  mode  to  transit,  auto  travel  time  is 
also  important. 


COVERAGE 

Many  factors  and  important  measures  may  influence  the 
coverage  and  assessment  of  coverage  for  a specific  transit 
system.  To  illustrate,  the  impact  of  park-and-r ide  lots  on 
service  area  boundaries  may  cause  the  potential  extension 
and/or  redefinition  of  the  service  area.  An  examination  of 
those  measures  associated  with  increasing  transit  coverage 
indicates  that  there  are  three  major  aspects,  namely, 
spatial  coverage,  temporal  coverage,  and  access  potential  to 
the  system.  Some  of  the  derived  measures  are  dependent  upon 
site-specific  measures  discussed  earlier  in  these 
guidelines. 

It  should  be  noted  that  market  share  has  been  defined 
as  the  number  of  person  trips  made  using  the  SMD  service 
divided  by  the  number  of  person  trips  that  could  be  made 
using  the  service.  For  projects  where  the  names  of  SMD 
service  users  are  kept  on  file,  it  may  be  appropriate  to 
compute  an  additional  measure,  the  percent  of  service  area 
target  population  using  the  SMD  service. 


RELIABILITY 

The  next  SMD  Program  objective  is  the  improvement  of 
transit  reliability.  In  this  area  it  is  important  to 
differentiate  among: 

(1)  Adherence  of  transit  vehicles  to  schedules 
(dependability) ; 

(2)  Reliability  of  both  vehicles  and  support 
equipment;  and, 

(3)  Maintainability  of  both  vehicles  and  support 
* equipment. 
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The  assessment  of  adherence  to  schedule  is  fairly 
straightforward.  The  determination  of  measures  for 
reliability  and  maintainability  associated  with  transit 
vehicles,  however,  requires  a clear  definition  of  what 
constitutes  a failure  and  a distinction  between  preventive 
maintenance  (scheduled)  and  repair  (unscheduled 
maintenance) . 


PRODUCTIVITY 

Transit  productivity  describes  how  a system  and  its 
resources  are  allocated  and  managed  for  purposes  of 
providing  an  effective  transit  system.  In  essence, 
productivity  improvements  come  about  through  one  or  both  of 
the  following:  increased  level  of  service  provided  per  unit 

of  cost,  and  increased  number  of  passengers  carried  per  unit 
of  service  provided. 

Transit  productivity  is  depicted  in  Exhibit  8 as  an 
input-process-output  system. 


EXHIBIT  8.  TRANSIT  PRODUCTIVITY  AS  AN 
INPUT-PROCESS-OUTPUT  SYSTEM 
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The  system  input  consists  of  available  resources  such  as 
vehicles,  personnel,  facilities,  and  associated  cost.  The 
manner  in  which  these  resources  are  used  is  the  system 
process.  This  process  is  influenced  by  the  transit 
management  policies  and  the  manner  in  which  passengers 
utilize  the  transit  system.  The  next  step  is  to  assess  the 
influence  of  the  associated  resources  and  management 
policies  by  collecting  output  in  the  form  of  various 
productivity  measures. 

Productivity  measures,  as  given  in  Exhibit  7,  are 
essentially  organized  in  terms  of  the  above-mentioned 
elements:  input  (what  is  being  put  on  the  street  and  at  what 
cost) , process  (how  are  the  resources  being  managed  and  at 
what  cost)  and  output  (how  do  resources  relate  to  passenger 
utilization  and  at  what  cost) . 


TRANSIT  DEPENDENT 

The  SMD  Program  objective  of  improving  service  for  the 
transit  dependent  essentially  comprises  a combination  of  the 
first  four  SMD  objectives  in  the  context  of  one  or  more 
special  target  groups  (handicapped,  elderly,  jobless,  etc.). 
Accordingly,  many  of  the  measures  appearing  in  the  fifth 
section  of  Exhibit  7 are  similar  or  identical  to  measures 
presented  in  preceding  sections. 

Since  the  most  significant  transportation- related 
problem  of  these  target  groups  is  typically  limited 
mobility,  measures  relating  to  coverage  will  be  relevant  for 
most  evaluations.  In  particular,  it  will  be  desirable  to 
(1)  measure  increases  in  the  target  group  trip  production 
rates  by  trip  purpose,  and  (2)  record  perceptions  by  the 
target  group  and  others  regarding  social  gains  from  the 
increased  transit  coverage  and  attendant  mobility  changes. 
Although  projects  addressing  the  fifth  SMD  Program  objective 
may  not  explicitly  address  travel  time,  reliability,  and 
productivity  issues,  an  examination  of  the  impact  of  a new 
or  expanded  service  on  these  areas  may  be  needed  in  order  to 
obtain  a full  assessment  of  the  demonstration. 

The  Glossary  contains  definitions  relevant  to  the 
transit  dependent.  Revisions  to  these  definitions,  and 
possibly  to  the  list  of  measures  applicable  to  specific 
target  groups,  may  occur  as  information  becomes  available 
from  an  ongoing  research  effort  within  UMTA  on 
transportation  problems  of  the  transportation  handicapped. 
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OTHER  BASIC  MEASURES 


The  last  section  of  Exhibit  7 contains  measures  which, 
while  not  directly  applicable  to  the  five  SMD  Program 
objectives,  might  be  relevant  in  the  context  of  local  or 
national  concerns.  Energy  consumption  and  air  pollution  are 
two  issues  which  in  some  cases  constitute  the  direct  impetus 
for  transportation  improvements  undertaken  at  a 
demonstration  site.  Land  use  patterns  and  the  community 
attitudinal  profile  are  examples  of  measures  relating  to  the 
longer  term,  more  indirect  impact  of  SMD  innovations. 

It  is  anticipated  that  the  group  of  measures  in  this 
section  will  be  expanded  and  refined  over  the  course  of  the 
SMD  Program  to  reflect  the  growing  number  of  issues  which 
are  relevant  to  individual  demonstrations  and  the  evolving 
consensus  of  opinion  as  to  how  these  issues  should  be 
addressed. 


2.  Data  Collection/Derivation  Technigues 

Once  the  relevant  measures  for  project  evaluation  have 
been  determined,  it  is  necessary  to  identify  appropriate 
collection  or  derivation  techniques.  Collected  measures  can 
be  obtained  through  the  following  four  basic  methods: 

(1)  By  measurements,  using  various  instruments,  such 
as  stopwatches,  odometers,  and  speedometers.  The 
accuracy  of  the  recorded  data  is  a function  of  the 
accuracy  of  the  measuring  instrument  itself. 
Typical  measurements  include  travel  times  and 
vehicle  velocities. 

(2)  By  counts  or  observations  involving  tallies  either 
from  discrete  digitized  recording  equipment  or 
from  counts  made  by  individuals.  Typical  counts 
would  be  numbers  of  passengers  in  vehicles. 

(3)  By  surveys  or  interviews  which  provide  information 
relative  to  the  individual  being  questioned,  said 
information  to  include  such  items  as  origin, 
destination,  income  level,  previous  travel  modes, 
observations  of  how  the  service  is  functioning, 
and  attitudes  towards  transit  amenities. 

(4)  By  searching  records  such  as  those  available 
through  the  transit  system,  local  grant  recipient, 
and  other  local  planning  agencies  and  Census 
records. 
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Derived  measures  can  be  calculated  either  through  the  use  of 
simple  arithmetic  processes  or  special  analytic  models. 

This  category  of  measures  builds  upon  basic  data  collected 
through  some  of  the  above  means.  An  illustration  of  the 
former  type  of  derived  measure  might  be  dividing  passengers 
per  day  by  vehicle  miles  per  day  to  obtain  passengers  per 
vehicle  mile.  Examples  of  the  latter  type  of  derived 
measure  might  be  the  use  of  a time-delay  curve  to  estimate 
vehicle  speeds  or  the  calculation  of  reductions  in  fuel 
consumption  and  air  pollution  based  on  changes  in  traffic 
volumes.  At  present,  there  are  no  preferred  models  for 
calculating  the  derived  measures  included  in  Exhibit  7. 

In  view  of  the  large  number  and  variety  of  measures  in 
Exhibit  7 and  the  even  larger  number  which  are  likely  to 
arise  durinq  the  course  of  the  SMD  Program,  it  would  be  very 
difficult  to  specify  in  these  guidelines  a preferred  method 
of  data  collection  for  each  measure.  Moreover,  given  the 
rather  limited  experience  to  date  in  conducting  transit 
demonstrations,  it  would  be  inappropriate  to  attempt  to 
choose  a set  of  "best'*  methods  from  among  the  techniques 
alreadv  tried;  rather,  it  is  desirable  to  encourage  the 
continual  development  and  implementation  of  novel  techniques 
with  potential  for  increasing  the  efficiency  or  accuracy  of 
evaluations.  Finally,  there  is  really  no  requirement  for 
uniformity  among  data  collection  techniques,  but  rather 
there  is  a need  for  consistency  and  comparability  of  the 
data  obtained  by  these  collection  techniques.  The 
techniques  can  differ  from  project  to  project,  so  long  as 
they  are  comparable  in  terms  of  accuracy  and  yield  data  in  a 
form  suitable  for  analysis  both  within  the  project  and  among 
projects. 

For  the  above  three  reasons,  it  is  not  the  intent  here 
to  prescribe  a standardized  approach  to  data  collection. 
However,  it  is  appropriate  to  discuss  the  potential 
applicability  of  some  of  the  specific  techniques  listed  in 
Exhibit  7,  drawing  where  possible  from  previous  experience 
related  to  UMTA  demonstrations. 

Exhibit  9 illustrates  the  range  of  techniques  employed 
for  selected  measures  in  four  recent  UMTA  projects. 

Specific  comments  on  these  techniques  and  general 
recommendations  applicable  to  collecting  the  measures 
follow: 

(D  Travel  time,  speed,  and  vehicle  volume  data 

collection  techniques  can  range  from  manual  to 
automatic.  In  general,  automatic  techniques  are 
more  costly  than  manual,  and  thus  are  cost- 
effective  only  where  the  magnitude  of  data 
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EXHIBIT  9.  DATA  COLLECTION  TECHNIQUES  FOR  SELECTED  MEASURES 
USED  IN  RECENT  UMTA  DEMONSTRATION  PROJECTS 
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requirements  or  some  other  special  circumstances 
warrant  their  use. 

Some  of  the  more  sophisticated  automatic 
procedures  are  subject  to  reliability  problems. 
Failure  of  these  devices  can  cause  loss  of  vital 
data,  which  will  in  turn  delay  the  evaluation,  and 
considerably  increased  costs.  In  addition,  the 
measurement  accuracy  pf  automatic  or  semi- 
automatic devices  may  be  questionable, 
particularly  if  they  have  not  been  used 
extensively  before.  In  cases  where  definitive 
information  on  device  accuracy  is  not  available, 
it  is  essential  to  confirm  the  accuracy  of 
automatically  collected  data  by  periodic  use  of 
manual  devices. 

Simple  manual  devices  can  be  deployed  so  as  to 
maximize  utilization  of  roadside  personnel.  For 
example,  in  the  Shirley  Hiqhway  project,  the  use 
of  special  counters  by  each  observer  permitted 
keepinq  track  of  the  auto  occupancy  of  each 
vehicle  counted,  with  the  result  that  two  measures 
were  obtained  at  once.  The  Miami  project  is  also 
using  special  manual  devices  to  obtain  vehicle 
counts  and  occupancy  data  simultaneously. 

(2)  Experience  with  past  demonstrations  has  shown  that 
there  is  a lack  of  consistency  between  passenger 
counts  recorded  by  transit  personnel  and  counts  by 
on-board  or  roadside  observers.  For  instance,  in 
the  Seattle  project  it  was  found  that  bus  drivers 
tend  to  overestimate  the  passenger  load  while  on- 
board and  on-street  counters  tend,  on  the  average, 
to  be  consistent  with  each  other.  If  transit 
personnel  are  to  record  such  data,  it  is  essential 
that  verifications  be  made  during  the  project  to 
detect  any  potential  bias  or  unusual  variability 
in  this  data. 

(3)  In  utilizing  transit  system  records  and  service 
area  records,  such  as  census  data,  it  is  critical 
to  ascertain  accuracy  of  this  data.  Usually, 
discussions  with  personnel  who  initially  record 
this  data  will  provide  an  assessment  of  accuracy. 
Further,  where  special  data  is  collected  for  the 
project  by  a local  organization,  monitoring 
procedures  will  be  established  to  assure  that  no 
modifications  in  procedures  or  notations  have 
occurred  which  might  have  an  impact  on  the 
evaluation  process. 
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Great  care  should  be  taken  in  utilizing  fare  box 
revenue  as  a basis  for  arriving  at  passenger 
counts.  The  adequacy  of  this  approach  has  been 
shown  to  depend  heavily  on  fare  structure, 
considering  prices,  zones,  time  of  day,  and 
revenue  versus  non-revenue  loads. 

(4)  Demographic,  behavioral,  and  attitudinal  data  on 
users  and  non-users  of  the  demonstration  service, 
as  well  as  attitudinal  information  from  transit 
operators,  can  be  collected  through  a wide  variety 
of  survey  and  interview  techniques,  with  varying 
degrees  of  respondent  cooperation,  accuracy,  and 
cost.  In  view  of  the  large  amount  of  documented 
survey  experience  relating  to  both  transportation 
and  general  market  research  contexts,  and  in  view 
of  the  large  anticipated  role  of  surveys  in  SMD 
evaluations,  an  entire  appendix  has  been  devoted 
to  a discussion  of  survey  design  and  execution. 

In  evaluating  the  array  of  existing  and  potentially 
innovative  collection  techniques  relative  to  a particular 
measure,  some  of  which  are  included  in  Exhibit  7 as 
examples,  the  contractor  should  consider  factors  such  as  the 
cost  and  accuracy  of  each  method,  the  availability  of  local 
resources  to  implement  each  method,  the  ease  of 
implementation,  and  the  ultimate  data  analysis  requirements. 

With  respect  to  cost,  the  contractor  should  apply  sound 
judgment  in  determining  whether  the  anticipated  cost  of 
using  a particular  technique  is  justifiable  in  terms  of  the 
contribution  to  the  overall  project  evaluation  of  the 
specific  measure  being  collected.  Clearly,  the  total 
project  expenditure  for  data  collection  should  be  allocated 
amonq  individual  measures,  taking  into  account  each 
measure's  contribution  to  the  project  evaluation.  The 
contractor  should  make  special  note  of  any  data  item  which 
is  relevant  to  the  evaluation  but  whose  collection  cost 
appears  to  be  disproportionately  high  in  relation  to  other 
items. 

The  contractor  should  determine,  again  on  a judgmental 
basis,  whether  the  accuracy  of  a particular  technique  is 
consistent  with  the  accuracy  requirement  for  the  measure, 
which  in  turn  is  dependent  on  the  relative  importance  of  the 
measure.  A very  accurate  technique  is  probably  not 
warranted  for  a relatively  insignificant  measure,  especially 
if  that  technique  would  be  expensive  to  implement.  In 
addition,  a high  degree  of  accuracy  for  some  measures  may  be 
inconsistent  with  a lesser  degree  of  accuracy  for  others. 
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The  contractor  should  also  evaluate  alternative 
techniques  in  light  of  the  available  local  resources  — 
manpower  as  well  as  equipment.  An  attempt  should  be  made  to 
utilize  existing  equipment  wherever  feasible,  rather  than 
opting  in  favor  of  techniques  which  require  the  purchase  of 
new  equipment  (which  might  not  be  needed  by  the  locality 
after  the  demonstration  evaluation) . 

The  contractor's  Evaluation  Plan  should  contain 
justification  for  selecting'  the  particular  technique 
applicable  to  each  measure  in  terms  of  these  considerations. 
In  the  case  of  a novel  technique,  it  is  required  that  the 
contractor  demonstrate  acceptable  accuracy  before  it  can  be 
used  as  the  sole  source  for  data  collection.  It  is  further 
required  that  the  evaluation  contractor  document  his 
experience  with  those  data  collection  methods  employed  in  an 
evaluation,  as  explained  below  in  Chapter  IV.  As  this 
further  experience  develops,  TSC  will  make  this  information 
available  via  updates  to  the  Guidelines  document. 


C.  PLANNING  CONSIDERATIONS  RELATIVE  TO  DATA  COLLECTION  AND 

ANALYSIS 

The  preceding  section  contained  guidelines  relative  to 
specifying  appropriate  measures  and  collection/derivation 
techniques  for  addressing  SMD  Program  objectives  and  other 
project  objectives  and  issues.  This  section  completes  the 
discussion  of  evaluation  planning  activities  with  general 
guidelines  for  data  collection  and  analysis  procedures.  The 
material  in  this  section,  while  intended  to  be  applied  to 
individual  measures  selected  for  inclusion  in  the 
evaluation,  is  presented  in  a general  context.  The 
following  topics  are  included:  basic  data 

collection/analysis  design,  measure  stratification,  sampling 
requirements,  and  the  timing  of  data  collection. 


1 . Basic  Data  Col lection/ Analysis  Design 

A significant  aspect  of  the  evaluation  process  for  SMD 
demonstrations  is  determining  the  basic  data  collection/ 
analysis  design  to  be  employed  relative  to  specific  project 
objectives.  There  are  a great  variety  of  potential 
approaches,  ranging  from  an  "after-only"  design  (a  one-shot 
case  study  approach  involving  a single  set  of  measurements 
taken  after  the  demonstration  is  operational)  to  a "before- 
after  with  control  group"  design  (involving  a comparison  of 
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multiple  measurements) . A comprehensive  discussion  of  the 
specific  utility  and  the  relative  pros  and  cons  of  the 
various  design  approaches  can  be  found  in  Chapter  4 of 
Charles  River  Associates,  Measurement  of  the  Effect  of 
Transportation  Changes , September,  1972,  and  in  Donald  T. 
Campbell  and  Julian  C.  Stanley,  Experimental  and  Quasi- 
Experimental  Designs  for  Research , Rand  McNally  and  Company, 
Chicago,  1968.  The  information  which  follows  is  intended  to 
discuss  the  relative  advantages  of  various  approaches  in  the 
context  of  the  SMD  program  and  to  highlight  the  major 
considerations  involved  in  selecting  the  appropriate  design 
for  each  SMD  evaluation,  or  for  individual  measures  included 
in  the  evaluations. 

In  general,  a single  set  of  measurements  (for  example, 
taken  while  the  demonstration  is  in  operation)  will  be 
insufficient  for  assessing  the  impact  of  the  demonstration, 
since  it  will  not  provide  any  yardstick  with  which  to 
interpret  the  measurements.  It  is  recommended,  therefore, 
that  every  data  collection/analysis  program  be  structured 
around  some  form  of  comparison.  If  such  an  approach  is  for 
some  reason  infeasible,  the  contractor  must  indicate  the 
reason (s)  in  the  Evaluation  Plan. 

Given  that  the  basic  data  collection/analysis  design 
will  generally  be  in  the  form  of  a comparison  of  multiple 
measurements,  the  next  question  to  be  considered  is  what 
types  of  comparison  are  appropriate.  The  two  main  forms  of 
comparison  are  before  vs.  after  and  test  vs.  control.  In 
a before-after  comparison,  a given  measure  is  collected  on  a 
given  system  element  before  the  experimental  or  exemplary 
demonstration  technique  is  instituted  and  then  again  while 
the  technique  is  operational.*  In  a test-control 
comparison,  a given  measure  is  collected  on  a system  element 
which  has  been  affected  by  the  introduction  of  a 
demonstration  technique  (test  unit)  and  also  on  an 
equivalent  system  element  which  has  not  been  similarly 
treated  (control  unit) . Each  type  of  comparison  is  somewhat 


*As  is  discussed  below,  a before-after  comparison  does  not 
necessarily  imply  a single  measurement  before  the  demon- 
stration is  implemented  and  another  measurement  while  it  is 
operational.  Rather,  this  type  of  comparison  can  take  the 
form  of  a series  of  measurements  prior  to,  during,  and  after 
the  operational  phase  of  the  demonstration.  If  the  project 
is  implemented  in  stages,  there  will  be  a series  of  measure- 
ments corresponding  to  each  stage. 
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limited:  the  before-after  comparison  fails  to  show  what 

portion  of  the  change  in  the  measure  is  due  to  exogenous 
factors  unrelated  to  the  demonstration;  the  test-control 
comparison  shows  the  difference  between  "after"  measures  and 
hence  accounts  for  exogenous  factors,  but  fails  to  indicate 
the  degree  of  change  from  the  before  state  to  the  after 
state.  Accordingly,  it  is  desirable,  where  feasible,  to 
conduct  a before-after  comparison  in  conjuction  with  a test- 
control  comparison.  In  other  words,  the  data 
collection/analysis  design- should  if  possible  involve 
observation  of  both  a control  and  test  unit  before  and  after 
the  institution  of  the  treatment. 

To  make  the  foregoing  discussion  more  concrete, 
consider  a large  area  with  many  bus  routes  and  suppose  that 
a certain  fraction  of  them  are  treated  in  some  manner  -- 
i.e.,  a technique  is  implemented  which  can  be  expected  to 
reduce  bus  travel  time.  If  pre- treatment  and  post- treatment 
measures  of  travel  time  are  made  only  on  the  treated  routes 
and  a reduction  in  time  is  indicated,  there  is  no  way  of 
knowing  the  extent  to  which  the  improvement  is  attributable 
to  exogenous  factors  (for  instance,  a decrease  in  auto 
traffic  on  the  streets  where  the  buses  operate) . In  order 
to  account  for,  in  a quantitative  fashion,  these  known  or 
unknown  factors  which  have  arisen  during  the  interval 
between  the  before  and  after  measurements,  it  is  necessary 
to  make  before  and  after  measurements  of  bus  travel  time  on 
routes  which  are  comparable  to  the  test  routes  and  therefore 
susceptible  to  the  same  set  of  exogenous  factors.  The 
difference  between  the  travel  time  reduction  on  the  test  vs. 
control  routes  can  then  be  taken  as  the  true  change  due  to 
the  treatment.  To  make  these  statements  it  is  necessary  to 
be  fairly  confident  that  conditions  affecting  both  control 
and  experimental  units  are  "reasonably’'  similar  --  a 
requirement  which  is  sometimes  difficult  if  not  impossible 
to  assure. 

To  reiterate,  the  proper  use  of  the  combined  before- 
af ter/test-control  approach  guarantees  to  the  greatest 
extent  that  any  observed  improvement  is  indeed  due  to  a 
demonstration  technique.  Thus  the  contractor  should  employ 
both  types  of  comparisons  wherever  appropriate  and  feasible. 

The  determination  of  appropriateness  of  the  combined 
approach  involves  a consideration  of  the  nature,  scope,  and 
time  span  of  the  demonstration  project.  Regarding  the 
nature  of  the  project,  the  key  issue  is  whether  it  is 
exemplary  or  experimental  (see  Chapter  II,  Section  A for  a 
description  of  these  two  SMD  categories) . In  general,  the 
exemplary  demonstrations  are  less  investigative  than  the 
experimental  demonstrations — that  is,  there  is  a much 
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greater  a priori  knowledge  of  the  probable  effects  of  the 
techniques  or  services  being  applied  than  in  the  case  of 
experimental  demonstrations.  Thus,  the  levels  of  accuracy 
and  capability  for  hypothesis  testing  which  are  inherent  in 
a combined  before-after/test-control  approach  may  not  be 
required;  before-after  measurements  on  just  the  test  units 
may  be  sufficient. 

Regarding  the  scope  of  the  project,  exemplary 
demonstrations  will  generally  be  implemented  on  a 
significantly  larger  scale  than  the  experimental 
demonstrations,  since  they  are  aimed  at  illustrating  the 
benefits  of  certain  techniques  and/or  services  in  a full- 
fledged  operational  environment,  whereas  the  experimental 
projects  are  aimed  at  determining  the  feasibility  of  a 
particular  service  or  technique  in  a simulated  or 
miniaturized  operational  environment.  The  larger  the 
geographic  area  encompassed  by  or  affected  by  the  project, 
the  greater  the  possibility  that  no  control  units  can  be 
identified  (i.e.,  the  entire  population  is  composed  of  test 
units)  . 

Regarding  the  time  span  of  the  project,  no 
generalizations  can  be  made  with  respect  to  exemplary  vs. 
experimental  demonstrations,  since  projects  will  vary  in 
length  depending  on  a variety  of  other  factors.  As  a 
general  rule-of-thumb,  the  desirability  of  the  combined 
before-after/test-control  approach  increases  with  the  time 
span  of  the  project,  since  this  approach  reveals  endogenous 
as  well  as  exogenous  changes  occurring  over  the  project's 
duration. 

The  determination  of  feasibility  of  the  combined 
approach  involves  questions  of  data  availability  and  project 
timing.  If  there  is  a known  deficiency  in  either  type  of 
comparison,  then  only  the  valid  comparison  should  be 
employed;  it  is  generally  better  to  do  without  a before 
observation  or  a control  observation  than  to  settle  for 
unsuitable  before  or  control  data.  In  the  event  that  only 
one  type  of  comparison  is  feasible,  there  are  alternative 
techniques  and  precautionary  measures  available  to  the 
contractor  to  compensate  for  the  absence  of  the  other  type 
of  comparison. 

If  no  control  group  exists  (e.g.,  if  the  demonstration 
affects  the  entire  population  of  observation  units,  making 
each  one  a test  unit)  or  if  no  suitable  group  can  be  found 
(each  test  unit  is  unique) , then  the  contractor  should  be 
especially  observant  throughout  the  evaluation  period  of 
possible  exogenous  factors  which  might  influence  the 
interpretation  of  project  results.  Any  statistics  regarding 
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the  before  vs.  after  change  due  to  the  demonstration 
technique  should  be  examined  very  carefully  in  the  context 
of  these  observed  exogenous  factors,  and  any  conclusions 
based  on  such  statistics  should  be  qualified  accordingly. 

If,  due  to  project  timing,  there  is  no  opportunity  to 
perform  before  measurements,  or  if  it  is  known  beforehand 
that  the  units  to  be  observed  will  undergo  considerable 
change  between  the  before  and  after  periods,  the  contractor 
should  attempt  to  obtain  surrogate  data  for  the  before 
period.  Possible  sources  of  surrogate  data  would  include 
(1)  surveys  conducted  after  the  demonstration  is  operational 
which  question  people  about  conditions  or  their  behavior 
prior  to  the  implementation  of  the  demonstration  technique 
and  (2)  demographic  and  travel  data  collected  by  the  local 
highway  department,  planning  agency,  or  transit  operator 
some  time  prior  to  the  demonstration.  The  surrogate  data 
can  be  used  to  provide  some  indication  of  the  magnitude  of 
the  before- after  change  experienced  by  the  test  and  control 
groups. 

In  using  the  before- after  and/or  test-control  approach, 
one  of  the  key  steps  is  identifying  comparable  units.  To  as 
great  an  extent  as  possible,  the  units  observed  for  the 
before  case  must  be  equivalent  to  the  units  observed  for  the 
control  units.  Peturning  to  the  previous  example  of  bus 
routes,  before-after  comparability  is  not  a difficult 
problem,  since  the  same  routes  can  be  observed  for  both  time 
periods.  The  only  note  of  caution  is  that  the  routes  should 
be  unchanged  (with  respect  to  length,  number  and  location  of 
stops,  etc.)  from  one  measurement  period  to  the  next. 

Test-control  comparability,  on  the  other  hand,  raises 
some  interesting  problems.  Theoretically  the  test  and 
control  units  should  be  as  nearly  alike  as  possible  to  rule 
out  any  chance  of  the  observed  change  being  a result  of 
something  other  than  the  experimental  treatment.  Test  and 
control  units  should  be  chosen  which  are  similar  in  terms  of 
variables  assumed  to  be  related  to  the  particular  measure. 
Again  using  the  example  of  bus  routes  and  the  measure  travel 
time,  matching  of  test  and  control  routes  could  be  done  on 
the  basis  of  such  descriptors  as  route  length,  total  trips 
along  the  route,  peak  headway,  and  average  speed. 

TSC's  Evaluation  Framework  will  generally  suggest  the 
basic  data  collection/analysis  design  to  be  employed  for 
each  project  as  a whole  or  for  particular  measures  --  i.e., 
before-after  comparison,  test-control  comparison,  both  types 
of  comparison,  or  a single  set  of  measurements.  The 
contractor  should  determine  the  feasibility  of  such 
suggestions  in  terms  of  the  data  availability  and  time 
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framework  of  the  particular  project  and  site.  The 
contractor*  s Evaluation  Plan  should  then  elaborate  on  the 
approach  finally  selected  for  each  measure,  indicating 
information  such  as  the  specific  units  chosen  for  the 
control  and  test  groups. 


2 . Measure  Stratification 

Measure  stratification  refers  to  the  categorization  of 
individual  measures  for  collection/derivation  and/or 
analysis  purposes.  Examples  of  measure  stratification  are: 

(D  Peak  versus  off-peak  time  periods 

(2)  Day  of  the  week 

(3)  Revenue  (in-service)  versus  non-revenue  service 

(4)  Waiting,  access,  transfer  and  in-vehicle  travel 
times 

(5)  Fixed  route  versus  demand  responsive. 

Measure  stratification  improves  the  guality  of  the 
evaluation  by  allowing  an  assessment  of  how  changes  in 
measures  relate  to  the  stratification  categories,  hence 
increasing  the  transferability  of  results. 

Whereas  collection  of  an  unstratified  measure  provides 
only  a single,  average  reference  point,  the  use  of  a 
stratified  measure  provides  a series  of  reference  points, 
each  of  which  may  be  significant  to  the  analysis  and 
interpretation  of  results.  Knowledge  of  intercategory 
differences  in  results  enhances  transferability:  for 

instance,  if  a particular  demonstration  service  proves  to  be 
especially  beneficial  in  congested  areas  but  of  limited 
value  in  sparsely  traveled  areas,  then  other  sites 
considering  implementation  of  the  service  will  know  to  focus 
their  efforts  in  congested  areas. 

Stratification  can  take  the  following  forms: 

(1)  Categorization  of  a measure  into  additive 

components  — e.g.,  measuring  person  trip  time  in 
terms  of  trip  components  such  as  access  time, 
line- haul  time. 
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(2)  Categorization  of  a measure,  and  possibly  its 
components,  according  to  target  market, 
operational,  geographic,  or  time  categories  -- 
e.g.,  measuring  trip  time  for  peak  and  off-peak 
periods. 

(3)  Grouping  of  raw  values  of  a measure  into  class 
intervals,  with  class  intervals  determined  either 
before  or  after  data  collection  — e.g., 
determining  the  di-stribution  of  early,  late,  and 
on-time  arrivals. 

It  is  not  possible  at  this  time  to  present  a 
standardized  approach  to  be  used  for  each  measure.  Clearly, 
the  appropriate  type  and  level  of  stratification  depend  on 
the  particular  measure  and  on  the  characteristics  of  the 
site  and  project.  It  is  anticipated  that  a standardized 
approach  to  measure  stratification  will  evolve  over  time,  as 
experience  is  gained  regarding  the  incremental  benefits 
associated  with  each  degree  of  stratification.  However,  in 
order  to  provide  the  contractor  with  some  guidance  in  this 
area,  examples  of  possible  types  and  levels  of 
stratification  are  presented  below. 


a.  Categorization  of  a Measure  Into  Additive  Components 

This  form  of  stratification  involves  collecting  and 
reporting  data  separately  for  specific  components,  or  sub- 
breakdowns, of  a measure.  The  purpose  of  categorizing  in 
this  manner  is  to  single  out  the  effect  of  an  SMD  innovation 
on  these  specific  components.  Examples  of  this  form  of 
stratification  are  available  for  measures  relating  to  travel 
time,  reliability,  and  productivity. 

Person  transit  trip  time  for  fixed  route  systems  can  be 
broken  into  segments  as  depicted  in  the  following  diagram: 
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where: 

Segment  A = Access  time 

Segment  W = Waiting  time  for  first  vehicle  or  for 
subsequent  transfer 

Segment  T = In-vehicle  transit  time 

Segment  E = Egress  time 

t^  = Time  for  ith  trip  segment 

If  further  amplification  is  desired,  access  time  and 
egress  time  can  be  subdivided  into  walking,  riding,  and 
other  portions;  or  in-vehicle  transit  time  can  be  subdivided 
into  collection,  line-haul,  and  distribution  phases. 

In  the  case  of  demand-responsive  systems,  some  of  the 
trip  time  components  might  take  on  a different  definition: 
for  example,  access  time  would  be  zero,  and  waiting  time 
would  refer  to  the  difference  between  the  caller's  requested 
time  of  pickup  and  the  arrival  time  of  the  vehicle  at  the 
origin.  In  cases  where  the  caller  is  told  that  pickup  can 
only  be  made  later  than  the  requested  time,*  wait  time  can 
be  further  divided  into  the  time  between  the  requested 
pickup  time  and  the  promised  pickup  time,  and  the  time 
between  the  promised  pickup  time  and  the  arrival  time  of  the 
vehicle  at  the  origin.  This  latter  travel  time  component, 
is,  in  itself,  a basic  reliability  measure.  In-vehicle 
transit  time,  if  desired,  can  be  divided  into  the  direct 
routing  travel  time  (the  time  between  the  person's  origin 
and  destination  if  no  other  pickups  or  dropoffs  are  made) 
and  the  detour  travel  time  (the  time  spent  detouring  to  make 
other  pickups  and  dropoffs) . 

As  can  be  seen  in  Exhibit  7,  transit  vehicle  time  is 
always  to  be  broken  into  in-service  time  and  non- service 


*Due  to  the  potential  ambiguity  associated  with  requests  for 
immediate  service,  the  contractor  should  note  how  the 
particular  transit  operator  maintains  data  on  requested  and 
promised  pickup  times. 


3-42 


time.  However,  if  desired,  these  two  prime  categories  can 
be  further  divided  as  indicated  below. 

For  fixed  route  systems : 

In-service 

In  motion 
Loading 

Non-productive  — •'  waiting  for  lights,  metering, 
or  other  obstacles  to  motion 

Non-service 

Garage  to  first  service  point 
Last  service  point  to  garage 
Dead  turn-around  time 
Deadhead  time 
Other 

For  demand  responsive  systems: 

In-service 

In  motion  with  one  or  more  passengers  onboard 
In  motion  with  no  passengers  onboard  and 
in  the  act  of  picking  up  one  or  more 
passengers 
Loading 

Non- service 

Garage  to  first  pickup  point 
Last  dropoff  point  to  garage 
Between  first  pickup  point  and  last  dropoff 
point  with  no  passengers  onboard  and 
not  in  the  act  of  picking  up  one  or 
more  passengers 
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These  time  segments  are  depicted  in  the  following  diagram: 

ABCDEFGHI 

• • • • • • • • • 


where : 

Point  A = Garage 

B = First  pickup  point 

C = Dropoff  point  — no  passengers  on  vehicle 
but  driver  is  instructed  to  proceed 
immediately  to  pick  up  a passenger 
D = Pickup  point 

E = Dropoff  point  — no  passengers  on  vehicle 
and  there  are  no  requests  for  immediate 
pickup;  driver  is  instructed  to  proceed 
to  central  waiting  point 
F = Point  en  route  to  central  waiting  point  -- 

driver  is  instructed  to  proceed  immediately 
to  pick  up  a passenger 
G = Pickup  point 
H = Last  dropoff  point  of  day 
I = Garage 

Note  that  in  segments  BC  and  GH  pickups  and  dropoffs  are 
being  made  and  at  least  one  passenger  is  always  onboard. 

In-service  time  = t~  + t.,  + t.  + t,  + t_ 

l 3 4 6 7 

Non-service  time  = t^  + t^  + t 

Schedule  reliability  measures,  like  basic  measures 
relating  to  travel  time,  can  be  broken  into  individual  trip 
segments.  Equipment  reliability  measures  (for  instance, 
vehicle  delay  time  in  minutes  due  to  breakdown)  can  be 
stratified  by  type  of  breakdown,  possible  categories  being 
mechanical,  electrical,  tires,  etc.  The  decision  as  to 
whether  or  not  to  use  such  a breakdown  would  depend  on  the 
degree  of  emphasis  on  equipment  reliability.  This  breakdown 
would  probably  be  appropriate  only  in  projects  involving  new 
vehicle  types,  where  a detailed  record  of  failures  is 
necessary  for  design  modifications. 

For  operating  costs  of  SMD  demonstrations,  it  has  been 
decided  that  the  following  categorization  scheme  be  used: 
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Vehicle-related 
Driver- related 
Support-related 

Administrative  and  other  costs 

This  four-category  breakdown  of  costs  appears  to  be  an 
appropriate  type  and  level  of  stratification  for  measuring 
the  effect  of  SMD  innovations  on  operating  costs.  However, 
it  should  be  pointed  out  that  this  breakdown  will  not  be 
available  from  transit  company  records  but  will  have  to  be 
derived  by  aggregating  individual  cost  items. 

The  aggregation  of  cost  items  should  be  consistent  with 
project  FARE  expense  categories.*  Exhibit  10  is  a matrix 
showing  the  distribution  of  expense  object  classes  into 
functional  areas  under  the  FARE  system.  For  the  time  being, 
until  further  examination  of  the  FARE  classification  scheme 
and  definitions  is  completed,  it  is  recommended  that  the 
four  categories  of  operating  cost  be  derived  by  aggregating 
the  17  functional  categories  as  follows: 

Vehicle- related: 

050  Servicing  revenue  vehicles 

060  Inspection  and  maintenance  - revenue  vehicles 
070  Vandalism  repairs  - revenue  vehicles 

Dr iver -related: 

030  Revenue  vehicle  operation 


*UMTA,  in  cooperation  with  the  American  Transit  Association 
and  the  Institute  for  Rapid  Transit  (now  merged  into  the 
American  Public  Transportation  Association)  sponsored 
Project  FARE  (Uniform  Financial  Accounting  and  Reporting 
Elements)  to  develop  and  test  a consistent  reporting  system 
for  transit  company  financial  operating  data.  Although 
this  system  has  not  yet  been  implemented  on  an  industry- 
wide basis,  it  is  desirable  to  have  transit  data  that  is 
collected  under  the  SMD  Program  evaluations  be  consistent 
with  the  FARE  system.  Also,  more  recently  a similar 
uniform  reporting  system  has  been  developed  for  the  taxi 
industry  under  the  sponsorship  of  the  SMD  Program. 
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EXHIBIT  10 


FARE  REPORTING  SYSTEM 


WORKSHEET  FOR  FUNCTIONAL  DISTRIBUTION  OF  EXPENSE  OBJECT  CLASSES  - LEVEL  B.  A OGRE  CATED 


FUNCTIONS  FOR  DISTRIBUTING  EXPENSE  OBJECT  CLASSES 

Source:  Arthur  Andersen  & 

Co.,  Project  FARE  Task  IV 
Report,  Vol.  II:  Reporting 

System  Instructions,  Report 
No.  UMTA-IT-06-0034-73-6 , 
November  1973. 
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EXPENSE  OBJECT  CLASSES 

501.  LA BOB: 

01.  Operator*’  Salaries  4 Wages. 

* 

X 

x 

x 

02.  Ot Sep  Salaries  4 Wages 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

5 02.  FRINGE  BENEFITS: 

15.  Fringe  Benefits  Distribution 

X 

X 

X 

X 

X 

X 

X 

X 

X 

x 

X 

X 

X 

x 

x 

503.  SER7ICES : 

01.  Manageaent  Service  Fees 

X 

X 

X 

X 

- 

X 

03.  Prcfeasioaal  and  Technical  Services 

04.  Teaporary  Help 

■ 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

05.  Contract  Maintenance  Ser-rices 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

06.  Custodial  Services 

X 

07.  Security  Services 

X 

03.  Fro pul s ion  Fever 

X 

09.  Utilities  Other  Than  Propulsion  Paver... 

10.  0-uea  and  Subscriptions 

X 

X 

12.  Bridge.  T el  4 Bighvey  Tolls 

13.  Other  Services 

X 

X 

5 04.  MATERIALS  AST  SUPPLIES  CONSUMED: 

01.  Fuel  and  Lubricants 

X 

X 

02.  Tires  and  Tubes 

X 

X 

03.  Other  Materials  and  Supplies 

X 

X 

X 

X 

X 

X 

X 

X 

* 

X 

X 

X 

505.  CASUALTY  AST  LIABILITY  COSTS: 

01.  Preaiuas  for  Physical  Daaage  Insurance.. 

02.  Eecoveries  of  Physical  Daaage  Lasses 

04.  Payouts  for  Uninsured  P L 4 

? 0 Settleaen*. s 

X 

X 

X 

X 

X 

X 

05.  Provision  for  Uninsured  ? L 4 

? D Settleaents 

z 

06.  Payouts  for  Insured  P L 4 

? D Settleaents 

X 

07.  Recoveries  of  ? L 4 ? D Settleaents 

03.  Preaiuas  for  Other  Corporate  Insurances. 
09.  0c* e r Corporate  Losses 

X 

X 

10.  Eecaveries  of  Other  Corporate  Losses 

506.  LEASES  AND  RENTALS: 

and  E^uipnent 

02.  Passenger  Stations 

X 

03-  Passenger  Parking  Facilities 

I 

04.  Passenger  Revenue  Vehicles 

X 

05.  Service  Vehicles 

X 

06.  Opertting  Yards  or  Stations 

X 

07.  Engine  Houses,  Car  Shops  and  Garages 

03.  Fover  Generation  4 

Distribution  Facilities 

X 

X 

Control  Facilities 

X 

10.  Data  Processing  Facilities 

X 

11.  Revenue  Collection  and 

Processing  Facilities 

X 

12.  Other  General  Adainistrative  Facilities. 

503.  PROPERTY  RETIREMENT  WRITE-OFFS : 

01.  Property  Retirenent  Write-Off* 

X 

X 

X 

X 

X 

x 

510.  OTHER  TAXES: 

01.  Federal  Incoae  Tan 

X 

0?  S'lt*  Tex. 

I 

03.  Property  Tax 

X 

04.  Vehicle  Licensing  and  Registration  Fees. 

05.  Fuel  and  Lubricsat  Taxes 

X 

X 

X 

X 

06.  Other  Taxes 

X 

511.  EXPENSE  TRANSFERS: 

01.  F— ctica  Reclassifications 

X 

(X) 

X 

02.  Expense  Reclassifications 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

I 

X 

03.  Capitalisation  of  Nonoperating  Cost*.... 

512.  SUBSIDY  PAYMENTS: 

01.  Purchased  Transportation  Service 

X 

X 

I 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

z 
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Su  ppor  t - re 1 a t e d : 

080  Servicing  and  fuel  for  service  vehicles 
190  Inspection  and  maintenance  - service  vehicles 
100  Maintenance  - vehicle  movement  control  system 
110  Maintenance  - fare  collection  and  counting 

equipment 

120  Maintenance  - other  buildings,  grounds,  and 

equipment 

140  Operation  and  maintenance  of  electric  power 
facilities  - 

150  Ticketing  and  fare  collection 
180  General  function 

Administrative  and  other  costs: 

010  Administration  of  transportation  operations 
020  Scheduling  of  transportation  operations 
040  Maintenance  administration 
160  General  administration 

If  more  detail  is  required,  the  individual  rows,  columns,  or 
cells  of  the  matrix  can  be  used. 

Because  of  differences  in  current  accounting  practices 
within  transit  operations,  it  is  essential  that  any 
techniques  for  disaggregation  and  allocation  of  costs  be 
described  in  the  Evaluation  Plan.  In  addition,  because  of 
different  funding  and  accounting  mechanisms,  it  is  important 
to  review  in  depth  individual  transit  authority  practices.* 
Further,  demand  responsive  systems  will  require  somewhat 
different  approaches,  again  to  be  described  in  the 
Evaluation  Plan. 


*It  is  recognized  that  the  reporting  of  operating  costs 
should  be  carried  out  using  a consistent  time  framework  for 
reportina  periods.  Most  transit  operations  use  a monthly 
reporting  period.  This  causes  some  distortion  in  the 
monthly  cost  (revenue)  averages  due  to  the  variation  of  days 
of  revenue  service  between  months.  Drivers'  wages,  fuel  and 
oil  costs,  maintenance  costs,  and  revenues  are  all  directly 
proportional  to  the  actual  hours  of  service.  This  distor- 
tion can  be  minimized  by  moving  to  a four  week  reporting 
period  (as  has  been  done  in  the  Rochester  Service  and 
Methods  Demonstration  Project) , although  some  problems  exist 
even  here.  In  addition,  standardized  depreciation  tables 
should  be  developed  for  different  types  of  capital  facili- 
ties and  equipment. 
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b.  Categorization  of  a Measure  According  to  Target  Mark et. 

Operational,  Geographic,  or  Time  Categories 

The  primary  purpose  of  this  form  of  stratification  is 
to  evaluate  the  effect  of  SMD  innovations  in  different 
contexts.  As  in  the  case  of  categorization  into  additive 
components,  this  form  of  stratification  involves  collecting 
and  reporting  measures  separately  for  each  category. 

Examples  follow: 

Target  Market: 

Trip  purpose  --  work/nonwork 

User  group  --  commuters/noncommuters 

Mode  --  auto/transit/other 

Operational: 

Type  of  transit  service  — express/local 
Direction  of  traffic  flow  — inbound/outbound 
Type  of  thoroughfare  --  freeway /arterial 

Geographic: 

Within/outside  central  business  district 

Zones  with  different  demographic  characteristics 

Time : 

Peak/off-peak 
VJe  ekday/week-end 

Finer  stratification  in  the  above  examples  is  also 
possible.  For  instance,  within  the  target  market  category, 
the  trip  purpose  "nonwork"  can  be  divided  into  medical, 
social,  recreational,  etc.;  non-commuter  can  be  stratified 
into  elderly,  handicapped,  unemployed,  etc.;  and  mode  can  be 
divided  into  solo  driver  auto,  carpool  auto,  chauffeured 
auto,  and  specific  local  transit  service  options.  Types  of 
service  can  be  divided  into  local  feeder,  local  line- haul, 
and  express  line-haul,  and  further  divided  into  individual 
routes,  and  beyond  that  into  route  segments.  Time  of  day 
can  be  refined  into  the  four  Project  FARE  categories  (A.M. 
peak,  midday,  P.M.  peak,  night)  or  even  further  into  hour, 
half-hour,  or  15-minute  segments  within  certain  categories. 

In  general,  it  will  be  desirable  to  partition  collected 
data  into  various  target  market  categories,  since  most 
demonstrations  will  probably  consist  of  specific  innovations 
aimed  at  particular  user  groups.  The  decision  as  to  whether 
to  stratify  collected  data  by  operational  and  geographic 
categories  depends  on  the  nature  of  the  project  and  thus 
will  have  to  be  made  on  a case-by-case  basis.  However,  it 
is  recommended  that  the  minimum  time  of  day  stratification 
(peak. of f-peak)  be  used  for  every  measure,  since  many 
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transit  system  operating  characteristies  as  well  as  general 
traffic  conditions  vary  widely  between  peak  and  off-peak 
periods.  The  decision  as  to  stratification  of  data 
collection  within  the  peak  period  (i.e.,  morning  vs. 
evening  peak)  and  within  the  off-peak  period  (i.e.,  midday 
vs.  nighttime)  should  be  made  in  accordance  with  the  time 
of  SMD  service  operation  throughout  the  day  and  the 
variability  of  travel  conditions  and  other  relevant  factors 
between  the  different  categories.  It  is  important  to  note 
that  the  peak  period  may  be  a changing  period  depending  upon 
distance  from  the  CBD  and  type  of  transit  system  (e.g.,  in 
Minneapolis  it  was  found  that  the  peak  period  for  corridor 
analyses  had  to  be  expressed  as  a function  of  distance  from 
the  CBD) . 


c.  Grouping  of  Paw  Data  Into  Class  Intervals 

Measure  stratification  can  also  refer  to  the  grouping 
of  raw  data  into  intervals,  with  intervals  determined  before 
or  after  data  collection.  Whereas  the  first  two  forms  of 
stratification  involve  collecting  and  reporting  a measure 
separately  for  each  category  (e.g.,  change  in  travel  time 
during  peak  periods,  off-peak  periods),  this  type  of 
grouping  produces  a frequency  distribution  for  the 
particular  measure. 

Survey  data  on  traveler  behavior,  characteristics,  and 
attitudes  is  a good  example  of  pre-collection  determination 
of  intervals.  For  instance,  comparisons  of  users  and  non- 
users of  an  SMD  service  can  be  made  using  distributions  of 
such  measures  as  age,  income,  auto  availability,  and 
attitudes  toward  transit,  with  the  particular  response 
categories  of  each  measure  having  been  determined 
beforehand.  Appendix  A contains  recommended  response 
categories  for  selected  demographic  and  travel  behavior 
measures,  as  well  as  sample  questions  and  response 
categories  for  selected  attitudinal  data. 

Reliability  measures  provide  examples  of  intervals  that 
can  be  determined  after  data  collection.  The  difference 
between  scheduled  and  actual  arrival  time  at  an  access  point 
would  be  collected  in  its  raw  form  (i.e.,  each  vehicle's 
time  difference  in  minutes)  but  would  be  reported  as  a 
frequency  distribution.  It  is  recommended  that  the 
following  minimum  stratification  of  this  measure  be  used: 
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% early 

% on  time  (vehicles  arriving  within  + or  - 1 minute  of 
scheduled  time) 

% late 

The  contractor  should  be  aware  of  differences  in  transit 
company  standards  with  respect  to  schedule  adherence,  and 
the  potential  impact  on  data  collection  and  analysis 
procedures. 

Vehicle  delays  due  to  breakdowns  can  be  grouped 
according  to  the  following  minimum  stratification: 

% No  delay  (delay  of  1 minute  or  less) 

% Delayed 

% Total  disruption  of  service 

If  further  detail  is  desired,  the  late  category  under 
schedule  adherence  and  the  delayed  category  under  vehicle 
reliability  can  be  divided  into  categories  such  as:  1-5 
minutes  delay,  6-10  minutes  delay,  over  10  minutes  delay. 

The  basic  intent  of  grouping  is  to  summarize  the  raw 
data  without  masking  the  real  form  of  the  distribution  for  a 
given  measure.  In  addition,  the  extent  of  grouping  may  also 
depend  upon  the  specific  analyses  which  are  planned. 

Interval  grouping  can  be  used  in  conjunction  with 
either  of  the  two  forms  of  stratification  previously 
discussed.  For  instance,  person  trip  time  can  be  stratified 
into  components  (access  time,  etc.) , and  time  period  (peak 
vs.  off-peak) ; the  values  in  each  component  in  each  time 
period  can  be  grouped  into  5 or  1 0 minute  intervals  to 
obtain  a frequency  distribution. 

As  was  stated  above,  it  is  not  possible  in  these 
guidelines  to  present  a standardized  approach  to 
stratification  for  each  measure.  The  contractor  will 
therefore  have  to  rely  on  judgment  and  past  experience  to 
determine  which  types  of  variable  stratification  are  most 
likely  to  enhance  the  understanding  of  specific  areas  of 
demonstration  effectiveness  and  potential  application.  The 
contractor  should  plan  data  collection  activities  with  the 
finest  stratification  which  can  be  justified  as  appropriate 
for  the  demonstration  objectives.  Since  the  ultimate  sample 
size  will  be  directly  related  to  the  number  of  categories 
employed,  the  contractor  should  make  sure  that  the  available 
sample  units  are  sufficient  to  support  the  level  of 
stratification  deemed  desirable.  The  Evaluation  Plan 
developed  by  the  contractor  should  contain  justification  for 
the  type(s)  and  level  of  stratification  selected,  as  well  as 
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evidence  that  such  stratifications  are  feasible  from  the 
standpoint  of  data  and  sample  size  availability. 


3 . Sampling  Requirements 

Once  the  contractor  has  determined  the  basic  data 
collection/analysis  design  for  the  project  evaluation  and 
the  type(s)  and  level  of  stratification  for  each  measure, 
the  final  question  to  be  addressed  is  sampling  requirements. 

In  general,  measures  which  are  obtained  from  records 
(e.g. , transit  company  cost  and  operating  data)  will  be 
available  on  a continual  basis  over  the  entire  lifetime  of 
the  demonstration  project  and  will  not  require  sampling.  On 
the  other  hand,  measures  obtained  from  measurements,  counts, 
and  surveys  will  generally  not  be  available  on  a continual 
basis  but  will  have  to  be  collected  in  the  form  of  samples. 
There  may  also  be  situations  where  measurements  or  counts 
yield  continual  data  (e.g.,  real-time  traffic  surveillance 
systems)  , but  sampling  is  desired  in  order  to  reduce  data 
processing  expenses. 

When  collection  of  a particular  measure  involves 
sampling,  an  estimate  of  the  minimum  sample  size  must  be 
made  prior  to  the  initiation  of  the  data  collection  effort. 
In  estimating  sample  size  requirements,  the  objective  is  to 
have  a large  enough  sample  to  be  able  to  draw  valid 
inferences  about  the  population  from  which  the  sample  is 
drawn.  As  might  be  expected,  the  determination  of 
appropriate  sample  sizes  involves  trade-offs  between  the 
desired  level  of  precision  and  the  cost  of  data  collection. 
These  trade-off  decisions  in  turn  require  a determination, 
during  the  evaluation  planning  phase,  of  the  appropriate 
types  of  analyses  to  be  performed  (e.g.,  estimates  of 
population  parameters,  comparisons  between  two  or  more 
groups  of  sampled  data) . 

Appendix  B presents  specific  guidelines  relevant  to 
estimating  required  sample  sizes.  Included  in  the 
discussion  are:  (1)  references  to  statistics  books 

containing  sample  size  equations,  (2)  recommendations 
regarding  values  for  the  three  input  factors  in  the  sample 
size  equation;  and  (3)  suggestions  regarding  implementation 
of  the  field  data  collection  effort  based  on  the  calculated 
sample  size  values.  Appendix  B also  contains  a section  on 
the  basic  types  of  statistical  analysis  which  can  be 
performed,  and  recommended  confidence  levels  and  reporting 
formats . 
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The  contractor  should  follow  the  guidelines  in  Appendix 
B to  develop  appropriate  sample  sizes  for  each  measure.  The 
Evaluation  Plan  should  contain  the  sample  size  values,  along 
with  an  explanation  of  any  assumptions  or  special  procedures 
underlying  these  values  (e.g. , equations,  input  factor 
values  used) . 


4.  Timing  of  Data  Collection 

For  measures  based  on  sampling,  another  issue  to  be 
addressed  by  the  contractor  is  the  timing  of  data 
collection.  The  exact  periods  during  which  measures  are 
collected  has  a significant  effect  on  the  validity  and 
representativeness  of  evaluation  results,  since  the 
operation  and  effectiveness  of  a transportation  system  are 
sensitive  to  various  factors  associated  with  time.  Four 
basic  questions  arise  concerning  the  timing  of  data 
col lection: 

(D  The  appropriate  season (s)  of  the  year  and  day  (s) 
of  the  week  to  include  in  the  sample 

(2)  The  appropriate  duration  of  each  data  collection 
period 

(3)  The  proper  time  to  initiate  data  collection 

(4)  The  appropriateness  of  "one-shot"  vs.  periodic 
monitoring. 

The  particular  sea  son (s)  and  day(s)  depend  largely  on 
the  assumed  sensitivity  of  the  demonstration  technique  or 
service  to  each  time  unit.  If  it  is  deemed  appropriate  to 
assess  the  impact  of  the  SMD  project  under  reasonably  normal 
conditions,  data  collection  should  be  performed  during  the 
fall  and  spring,  when  weather  conditions  are  not  severe, 
schools  are  in  session,  and  few  people  are  on  vacation.  To 
the  extent  that  the  demonstration  evaluation  involves 
measures  related  to  travel  patterns  and  transit  usage,  the 
contractor  should  attempt  to  schedule  data  collection 
activities  during  those  two  seasons  which  are  most 
representative  of  normal  conditions.  On  the  other  hand,  if 
severe  weather  conditions  or  other  atypical  conditions  are 
an  inherent  feature  of  the  site  and  it  is  desirable  to 
examine  the  SMD  project  under  a full  range  of  possible 
conditions,  the  contractor  should  schedule  data  collection 
throughout  the  year  so  the  sample  observations  include 
extreme  as  well  as  normal  conditions. 
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If  a particular  SMD  service  operates  seven  days  a week, 
then  the  sample  of  days  should  include  both  weekdays  and 
week-end  days  (in  fact,  the  data  should  be  stratified  by 
weekday  vs.  week-end  day  to  highlight  the  difference  in 
service  viability  during  these  two  periods)  . Regarding 
which  day(s)  to  include  in  the  weekday  sample,  similar  logic 
applies  as  in  the  case  of  seasons.  If  the  aim  is  to  observe 
the  Droject  under  typical  weekday  conditions,  then  any 
day  (s)  with  abnormal  traffic  patterns  should  be  avoided.  In 
some  cities,  there  is  a difference  between  Monday  and/or 
Friday  conditions  vs.  Tuesday/Wednesday /Thursday 
conditions;  if  this  is  known  to  be  the  case  for  a particular 
demonstration  site,  then  data  collection  should  be  scheduled 
for  the  three  "typical”  days  rather  than  either  of  the 
atypical  days.  The  contractor  should  consider  the  special 
characteristics  of  the  demonstration  and  the  site  in 
deciding  on  which  days  are  appropriate.  If  a large  number 
of  days  are  going  to  be  involved,  and  there  is  no 
particularly  significant  distinction  among  days  of  the  week, 
then  a randomly  selected  sample  of  days  would  be  preferable. 

The  duration  of  each  data  collection  period  should  be 
determined  based  on  the  degree  of  day-to-day  variability  and 
on  the  required  sample  size.  If  the  particular  item  being 
measured  is  suspected  to  vary  in  behavior  from  one  day  to 
the  next,  then  the  data  collection  period  should  include 
several  days;  if  it  has  been  determined  that  only  Tuesdays, 
Wednesdays,  and  Thursdays  can  be  used,  then  several  weeks 
may  be  necessary  to  achieve  the  required  sample  of  days. 
Moreover,  if  the  sample  size  required  for  a particular 
variable  is  large,  then  several  days  of  data  collection  may 
be  appropriate  to  obtain  the  minimum  sample  of  observations. 

The  choice  of  initiation  time  for  each  data  collection 
period  is  dependent  on  a number  of  considerations,  the  chief 
one  being  that  the  "after"  data  collection  not  begin  until 
the  demonstration  has  settled  in  and  stabilized.  In  general 
it  will  probably  take  a few  months  for  a demonstrated 
technique  or  service  to  become  fully  operational,  with  all 
the  "bugs"  worked  out,  and  possible  behavioral  influences 
associated  with  the  innovation  diminished  or  eliminated. 

The  desire  is  to  achieve  a "steady  state"  for  the  system 
after  the  innovation  has  occurred.  The  time  to  achieve  this 
"steady  state"  undoubtedly  will  vary  from  project  to 
project.  Thus  data  collection  related  to  the  operational 
phase  of  the  demonstration  should  not  commence  until  these 
adjustments  and  modifications  are  completed.  Other  factors 
determining  the  initiation  date  for  data  collection  are  the 
desire  to  avoid  summer  and  winter  months  and  the  overall 
schedule  of  the  demonstration  project. 
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In  most  instances,  data  collection  will  be  performed 
for  discrete  phases  of  the  demonstration  project  — i.e. , 
before  the  project  is  implemented,  while  the  project  is 
operational,  and  possibly  after  the  project  is  terminated.* 
However,  if  demonstration  elements  are  by  nature  changing 
continually  or  if  it  is  expected  that  the  demonstration  will 
cause  gradual  but  continual  changes  in  user-  or  operator- 
related  measures,  then  a periodic  process  of  data  collection 
would  be  more  appropriate  than  merely  "before,"  "during," 
and  "after"  data  collection.  The  multitude  of  data  points 
obtained  from  a periodic  monitoring  process  will  make 
possible  the  examination  of  fuctional  relationships  either 
among  measures  of  interest  or  in  a time  series.  Moreover, 
monitoring  of  certain  measures  during  the  early  months 
following  introduction  of  the  innovation  (s)  may  be  useful  in 
determining  when  the  effects  have  stabilized  enough  to 
initiate  full-scale  data  collection.  It  should  be  noted 
that  if  periodic  data  collection  is  appropriate,  then  a 
sequential  analysis  procedure  (similar  to  control  charts) 
may  be  useful  to  permit  reductions  in  sampling  requirements. 

The  contractor's  Evaluation  Plan  should  indicate  the 
exact  timing  of  data  collection  for  each  measure  involving 
sampling.  This  information  should  be  presented  in  a 
schedule  which  also  shows  the  projected  implementation  dates 
for  the  various  elements  of  the  project. 


♦Post-demonstration  data  collection  would  only  be  performed 
if  there  was  a desire  to  see  whether  operation  of  the  demon- 
stration technique  or  service  for  a limited  period  had  led 
to  permanent  changes  in  people's  travel  patterns  or  attitudes. 
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CHAPTER  IV 


GUIDELINES  FOR  PERFORMING  EVALUATION  ACTIVITIES 


This  chapter  presents  suggestions  relative  to 
implementing  the  evaluation  of  an  SMD  demonstration.  During 
the  evaluation  implementation  phase  of  the  demonstration 
evaluation  process,  data  collection/analysis  relating  to 
site  characteristics,  quantitative  measures,  and  qualitative 
measures  is  undertaken  according  to  the  plans  and  procedures 
laid  out  in  the  Evaluation  Plan.  In  addition,  information 
is  gathered  relative  to  the  project's  operational  history 
and  exogenous  events  which  may  have  some  bearing  on  the 
project  outcome.  This  information  is  eventually 
incorporated  into  the  analysis  and  interpretation  of  project 
results. 

Contractor  functions  during  the  evaluation 
implementation  phase  include  monitoring  and/or  performance 
of  data  collection  activities,  data  reduction  and  analysis, 
subjective  analysis  of  information  relative  to  project 
issues,  and  synthesis  of  project  findings  into  a Final 
Evaluation  Report.  In  accordance  with  these  contractor 
functions,  this  chapter  of  the  guidelines  is  organized  into 
two  sections:  (1)  monitoring /performance  of  data 

collection;  and  (2)  data  reduction,  analysis,  and 
presentation.  The  recommended  content  and  organization  of 
the  various  contractor  reports  prepared  during  this  phase 
are  presented  in  Chapter  V. 

•During  this  phase,  the  contractor  must  maintain  a 
sensitivity  to  the  relationships  among  the  organizations 
involved  in  the  project  --  in  particular,  the  grantee,  UMTA, 
and  TSC  (see  Chapter  II,  Section  A) . The  contractor  must 
work  closely  with  these  groups  at  the  appropriate  times, 
while  maintaining  the  role  and  perspective  of  an  external, 
objective  organization  assessing  the  impact  of  the 
demonstration. 


A.  MONITOR ING/PER FOR MANCE  OF  DATA  COLLECTION 

Since  much  of  the  data  required  for  SMD  evaluations 
will  be  unavailable  from  pre-existing  data  bases  and 
secondary  sources,  each  demonstration  will  undoubtedly 
involve  significant  data  collection  efforts.  Given  the 
considerable  amount  of  time  and  money  which  will  be  spent  on 
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data  collection,  careful  management  and  overseeing  of  the 
data  collection  process  is  essential. 

The  contractor  is  responsible  for  seeing  that  data 
collection  is  performed  according  to  the  TSC/UMTA-approved 
Evaluation  Plan.  There  are  three  potential  alternatives 
associated  with  data  collection.  One  of  these  occurs  when 
the  grantee  collects  all  data  (under  SMD  and/or  local 
funding) , and  the  contractor  acts  in  a monitoring  role  to 
assure  the  quality  and  timeliness  of  data  collected,  as  well 
as  adherence  to  procedures  laid  out  in  the  Evaluation  Plan. 

A second  alternative  occurs  when  the  contractor  collects  all 
the  data,  and  coordinates  the  timing  and  performance  of 
these  activities  through  the  grantee.  The  third  and  most 
likely  possibility  is  one  in  which  both  grantee  and 
contractor  collect  various  elements  of  the  data. 

In  order  to  monitor  and/or  perform  the  data  collection 
activities  called  for  in  a given  evaluation,  the  contractor 
will  need  to  maintain  open  channels  of  communication  with 
the  site,  in  the  form  of  visits,  telephone  contact  and 
written  correspondence  with  the  appropriate  local  agencies, 
as  well  as  subscriptions  to  local  newspapers.  Where  day-to- 
day  contact  with  the  site  is  necessary,  the  contractor 
should  arrange  to  base  a member  of  the  firm  at  the  site. 

Whether  data  collection  is  being  performed  by  the 
contractor  or  by  the  local  grant  recipient,  the  contractor 
must  stay  closely  involved  in  all  phases  to  make  sure  that 
the  procedures  specified  in  the  Evaluation  Plan  are 
followed.  In  cases  where  the  grantee  or  other  local  agency 
is  collecting  data,  the  contractor  should  meet  frequently 
with  the  agency  to  discuss  progress  and  problems,  work  out 
solutions  to  the  problems,  and  observe  key  phases  of  field 
data  collection.  In  addition,  the  contractor  should 
occasionally  perform  independent  spotchecks,  especially  in 
the  case  of  measures  for  which  the  local  agency  has  limited 
experience  in  data  collection. 

The  contractor  is  expected  to  inform  TSC  of  the  status 
of  data  collection  in  its  Monthly  Evaluation  Progress 
Reports  (see  Chapter  V for  the  recommended  content  and 
organization  of  this  type  of  report) . Should  there  be  any 
unacceptable  degradation  of  quality  or  timeliness  of  data 
collected  by  the  grantee,  the  contractor  should  notify  TSC 
in  writing.  TSC  will  in  turn  take  steps  through  the  UMTA 
Project  Manager  to  rectify  the  situation. 

Over  and  above  monitoring  data  collection  activities, 
the  contractor  should  keep  abreast  of  the  status  of  the 
demonstration  project.  This  awareness  of  project 
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operational  status  is  important  so  that  (1)  data  collection 
activities  can  be  smoothly  coordinated  with  ongoing  project 
activities  (causing  miminum  disruption  of  day-to-day 
operations)  and  (2)  so  that  evaluation  results  can  be 
interpreted  in  the  context  of  project  history.  The 
grantee's  Quarterly  Project  Progress  Reports  to  UMTA/TSC 
(see  Chapter  V for  recommended  content  and  organization) 
will  be  a useful  source  of  information  on  the  project's 
operational  evaluation.  However,  the  contractor  is 
encouraged  to  obtain  a more  -detailed  account  of 
progress/problems  relative  to  implementing  and  operating  the 
demonstration  by  talking  with  the  grantee  at  the  site. 

In  addition  to  keeping  abreast  of  project  operations, 
the  contractor  should  be  continually  watching  at  the  site 
for  unexpected  (exogenous)  events  which  might  affect  the 
validity  of  project  results.  In  any  implemented 
demonstration,  no  matter  how  well  controlled  or  planned,  the 
possibility  remains  for  unexpected  events  to  occur  that  may 
have  an  impact  on  measures  of  the  project's  performance. 
These  unexpected  occurrences  are  classified  as  threats  to 
the  validity  of  the  demonstration. 

Unanticipated  developments  at  the  site  can  take  the 
form  of  temporary  events  such  as  a parking  lot  attendant 
strike  or  longer-term  phenomena  such  as  the  closing  of  a 
major  thoroughfare.  The  following  are  examples  of 
unexpected  factors  that  have  been  experienced  in  earlier 
demonstration  projects,  along  with  an  indication  of  the 
compensatory  action  taken  to  counteract  the  exogenous  event: 

Changes  in  employment.  There  were  thousands  of 
unemployed  in  Seattle  (Blue  Streak  Project)  due  to  the 
high  number  of  layoffs  in  the  aerospace  industry.  (No 
compensatory  action  was  taken.) 

Changes  in  freeway  traffic  volumes.  Shirley 
Highway  experienced  a shift  from  arterials  to  the 
freeway  upon  completion  of  new  lanes  and  sections. 
Minneapolis,  on  the  other  hand,  noted  a shift  to  the 
freeway  due  to  arterial  street  constructon  and  land 
developments  within  the  project.  Seattle's  "Blue 
Streak"  noted  volume  shifts  on  the  entrance  and  exit 
ramps  where  new  lanes  had  been  added  or  preferential 
treatment  was  given  to  buses.  Seattle  also  experienced 
a queuing  problem  onto  the  freeway  from  autos  that  were 
diverted  from  converted  ramps.  (An  adjustment  in 
queuing  sequence  was  made  where  necessary.) 
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The  national  energy  crisis.  Minneapolis 
experienced  a drastic  change  in  traffic  volumes  from 
auto  to  transit  during  the  energy  crisis.  Although  it 
cannot  be  determined  whether  the  shift  in  volumes  was 
directly  attributable  to  this  factor,  the  timing  of  the 
initiation  of  the  project  during  this  period  may  have 
had  some  impact  on  data  interpretation.  (Extended 
routes  and  an  increase  in  the  frequency  of  service  were 
the  immediate  modifications  made  to  facilitate 
transporting  such  large  number  of  people.  Also,  their 
marketing  campaign  was  modified  — slowed  down  — in 
view  of  the  large  numbers.) 

As  previously  noted,  the  use  of  a test-control  evaluation 
design  will  in  certain  cases  mitigate  the  impact  of  these 
unplanned  events  on  the  validity  of  the  project  results. 

The  contractor  is  responsible  for  informing  TSC  of  any 
unplanned  phenomena  which  arise  during  the  course  of  the 
evaluation.  The  contractor's  Monthly  Evaluation  Progress 
Report  should  describe  the  potential  effects  on  validity  of 
any  phenomena  noted,  as  well  as  propose  changes  in  the 
project  and/or  evaluation  to  compensate  for  the  unplanned 
occurrences . 

Although  data  collection  should  generally  proceed 
according  to  the  Evaluation  Plan,  there  may  be  instances 
where  modification  to  the  originally  planned  procedures  is 
warranted.  The  previous  paragraph  indicated  that  exogenous 
events  at  the  site  might  be  cause  for  modifying  the 
evaluation.  Two  additional  reasons  for  deviating  from  the 
planned  approach  are  discussed  below,  namely,  operational 
changes  in  the  project,  and  availability  of  improved 
evaluation  techniques. 

Operational  changes  in  the  project  can  come  about  as  a 
result  of  contractor  recommendations  (transmitted  in  the 
Monthly  Evaluation  Progress  Reports)  or  decisions  by  UMTA 
and  the  grantee.  Whatever  the  impetus  for  these  changes  in 
the  scope  or  operation  of  the  demonstration,  the  evaluation 
will  have  to  be  modified  accordingly.  The  contractor  is 
responsible  for  assessing  the  impact  on  the  evaluation  of 
any  forthcoming  or  proposed  operational  changes,  and 
recommending  appropriate  modifications  of  the  Evaluation 
Plan  to  TSC. 

As  new  data  collection  techniques  are  developed  in  the 
course  of  the  SMD  program,  it  may  be  appropriate  to  modify 
certain  aspects  of  a project's  Evaluation  Plan.  The 
contractor  will  have  to  assess,  on  a case-by-case  basis, 
whether  the  potential  benefits  of  the  new  techniques  are 
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sufficient  to  justify  modification  to  the  planned  evaluation 
activities,  and  then  recommend  the  appropriate  course  of 
action  to  TSC. 

In  order  to  further  the  state-of-the-art  of  transit 
demonstration  evaluation,  the  contractor  is  responsible  for 
performing  an  ongoing  assessment  of  data  collection 
procedures  used.  It  should  be  fairly  evident  that  no 
attempt  has  been  made  in  past  demonstrations  to  record  in  a 
consistent  manner  information  on  how  well  the  method  worked. 
The  evaluation  contractor  should  maintain  close  control  over 
data  collection  procedures  used  and  summarize  findings  with 
respect  to  each  technigue.  These  findings  will  include,  as 
a minimum: 

(1)  A narrative  description  of  how  the  collection 
procedure  was  planned  and  implemented. 

(2)  An  indication  of  areas  in  which  the  technique 
outperformed  expectation. 

(3)  An  indication  of  areas  in  which  the  technique  was 
deficient. 

(4)  If  available,  some  summary  of  the  inherent 
variability  in  collecting  project  measures  due  to 
the  technique  itself,  as  opposed  to  variability 
due  to  other  demonstration  factors. 

(5)  Actual  cost  for  implementing  the  technique, 
including  capital  costs,  and  all  operating  and 
support  costs. 

(6)  Where  two  techniques  have  been  employed  to  collect 
the  same  basic  measures,  cross-comparisons  and  a 
recommendation  as  to  which  technique  should  be 
used  in  similar  future  demonstrations. 

This  information  will  ultimately  be  incorporated  into  an 
Appendix  of  the  Final  Evaluation  Report. 


B.  DATA  REDUCTION,  ANALYSIS  AND  PRESENTATION 

The  contractor  is  responsible  for  performing  all  data 
reduction  and  analysis,  regardless  of  which  agency  has 
collected  the  data.  Data  reduction  involves  the  processing 
of  raw  data,  either  manually  or  using  a computer,  to  yield 
statistics  such  as  means,  standard  deviations,  ratios. 
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ranges,  and  frequency  distributions.  The  specific  statistic 
to  be  calculated  from  the  raw  data  will  depend  on  the  type 
of  measure  and  type  of  comparison  involved.  A few  examples 
are  given  below.  Quantitative  measures  such  as  travel  time 
and  vehicle  passenger  counts  will  be  processed  into  average 
values  for  each  level  of  stratification  used.  If  a 
comparison  of  two  time  periods  is  involved,  the  percentage 
change  from  the  earlier  to  the  later  period  will  be 
calculated.  Quantitative  measures  relating  to  schedule 
dependability  will  be  summarized  into  average  values  as  well 
as  standard  deviations,  with  comparisons  calculated  as 
ratios  of  standard  deviations.  Qualitative  measures 
obtained  through  surveys  will  be  handled  to  yield  frequency 
distributions  for  the  response  categories. 

Data  reduction  may  involve  the  use  of  statistical 
inference  techniques.  If  the  data  is  based  on  a 100%  data 
collection  effort  (i.e. , no  sampling),  then  exact  values  of 
the  statistics  listed  above  can  be  calculated.  However,  if 
the  data  has  been  obtained  by  sampling  (as  will  usually  be 
the  case) , results  cannot  be  presented  as  precise  values, 
since  there  is  a certain  probability  that  the  calculated 
values  are  different  from  the  true  population  values.  It  is 
recommended  that  data  based  on  samples  be  processed  into 
two-sided  confidence  intervals  using  two  confidence 
levels:  a=.01  and  a=.05.  Appendix  B presents  further 

guidelines  relative  to  calculating  confidence  intervals. 

The  contractor  should  arrange  for  smooth  transfer  of 
collected  data  from  the  collection  site  (e.g.,  buses, 
transit  company,  roadside  stations)  to  the  processing  site. 
Special  attention  should  be  paid  to  details  such  as  labeling 
and  dating  of  forms,  tapes,  etc.  to  make  sure  that  valuable 
data  is  not  lost  or  altered. 

The  basic  data  which  is  collected  during  a 
demonstration  project  should  be  maintained  either  on  punch 
cards  or  other  appropriate  storage  devices  (e.g.,  magnetic 
tape) . While  the  raw  data  may  not  be  immediately  utilized, 
it  should  remain  with  the  contractor  (or  eventually  TSC)  for 
potential  future  uses. 

Data  analysis  involves  the  interpretation  and  synthesis 
of  the  processed  data  and  other  information  to  draw 
conclusions  relative  to  the  attainment  of  project  objectives 
and  issues,  and  relative  to  project  transferability. 
Statistics  such  as  means  and  frequency  distributions  are 
carefully  examined  and  pulled  together  to  obtain  a 
comprehensive,  in  depth  understanding  of  the  effects  of  the 
demonstration,  and  the  underlying  reasons  for  observed 
changes.  The  contractor  must  apply  sound  judgment  as  well 
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as  knowledge  and  experience  relative  to  transit  system 
operations,  traffic  operations,  and  travel  behavior  in  order 
to  interpret  the  collected  data  and  place  it  in  proper 
persoective.  To  the  extent  possible,  the  results  of  the 
demonstrated  innovations  in  the  demonstration  site  should  be 
supplemented  by  an  assessment  of  the  influence  of  site- 
specific  and  exogenous  factors  on  project  outcome,  so  that 
conclusions  can  be  made  regarding  the  potential 
applicability  and  effects  of  implementing  the  demonstration 
in  other  sites  across  the  country.  In  order  to  further 
enhance  project  transferability,  the  analysis/synthesis 
phase  should  provide  a compilation  of  lessons  learned 
regarding  the  operation  of  the  demonstration. 

The  contractor  should  understand  and  be  aware  of  the 
importance  that  the  use  of  appropriate  statistical 
techniques  can  attach  to  the  analysis  and  interpretation  of 
project  results.  In  view  of  the  fact  that  most  aspects  of 
an  urban  transportation  system  tend  to  be  dynamic  and 
variable  from  hour- to- hour , day-to-day,  and  month-to-month, 
observed  differences  could  be  attributable  only  to  this 
inherent  variability  and  not  to  the  SMD  innovations. 
Furthermore,  factors  other  than  the  planned  and  controlled 
innovations  could  also  be  directly  related  to  the  observed 
changes  in  those  measures  being  collected.  It  is  important 
to  note  that  while  no  single  technique  exists  for  removing 
the  potential  influence  of  these  exoqenous  factors,  it  is 
possible,  by  careful  analysis,  to  at  least  point  out  the 
occurrence  of  such  events  and  create  an  awareness  for  those 
who  review  the  project’ s conclusions  and/or  recommendations. 
Hence,  it  is  important  to  be  able  to  specify  whether  the 
observed  differences  in,  for  example,  travel  time  are  within 
reasonable  bounds  of  expected  variability  inherent  in  the 
given  transportation  system,  or  whether  the  observed 
differences  cannot  be  accounted  for  just  by  system  random 
variability.  If  the  latter  case  were  true,  taking  into 
consideration  the  potential  external  influencing  factors, 
one  could  conclude  that  the  innovation  has  in  fact  provided 
a real  change  in  the  measures  being  considered.  It  is  to 
this  capability  for  making  valid  inferences  that  the 
specific  statistical  techniques  apply. 

Presentation  of  project  results  in  Annual  Project 
Status  Summaries,  Interim  Evaluation  Reports,  and  Final 
Evaluation  Reports  should  be  in  the  form  of  quantitative  and 
qualitative  exposition,  with  exhibits  such  as  tables, 
graphs,  and  bar  charts  serving  as  the  focus  for  narrative 
discussion.  In  no  instance  should  there  be  an  excessive 
narrative  describing  all  the  elements  of  an  exhibit.  This 
tends  to  be  redundant  and  masks  the  really  important 
findings. 
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Chapter  V provides  some  guidance  relative  to  overall 
content  and  organization  for  the  aforementioned  reports. 

With  respect  to  the  format  for  exhibits,  creative  techniques 
for  displaying  information  are  encouraged,  so  long  as  the 
information  is  presented  in  a clear  and  accurate  manner.  In 
order  to  provide  the  contractor  with  some  indication  of  the 
types  of  exhibits  that  are  acceptable,  some  examples  from 
recent  projects  are  presented  on  the  following  pages. 

Exhibits  11-17  are  clear  and  informative.  While  they 
do  not  present  detailed  information,  they  are  useful  in 
highlighting  the  project  findings  appearing  in  an  executive 
summary,  which  is  designed  to  convey  rapidly  to  the 
decision-maker  the  significant  conclusions  of  the  project. 
Back-up  exhibits  which  contain  significantly  more  detail 
will  be  contained  within  the  body  of  the  evaluation  report 
or  in  technical  appendices.  Exhibits  18-22  fall  into  this 
category. 

The  contractor  should  attempt  to  perform  data  reduction 
and  analysis  as  data  is  collected,  so  that  interim  results 
are  available  throughout  the  project  evaluation.  These 
interim  findings  will  not  only  satisfy  general  curiosity 
regarding  the  project’s  effects,  but  will  also  provide 
feedback  information  relative  to  ongoing  project  operations 
and  evaluation.  Examination  of  preliminary  evaluation 
results  may  suggest  opportunities  for  modifying  the  project 
and/or  evaluation  procedures  so  as  to  increase  the  utility 
of  the  demonstration  projects. 
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EXHIBIT  11 

DIAGRAM  SHOWING  THE  SERVICE  AREA  FOR 
THE  SEATTLE  BLUE  STREAK  PROJECT 


EXHIBIT  12 


SCHEMATIC  SHOWING  HIGHWAY  RAMP  VOLUMES 
FOR  THE  SEATTLE  BLUE  STREAK  PROJECT 
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EXHIBIT  13 


DIAGRAM  SHOWING  DISTRIBUTION  OF  PARK-AND-RIDE 
USERS  FOR  THE  SEATTLE  BLUE  STREAK  PROJECT 
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EXHIBIT  14 


SCHEMATIC  SHOWING  PASSENGER  VOLUME  FOR  THE 
SEATTLE  BLUE  STREAK  PROJECT 
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PERCENTAGE  OF  TOTAL  PERCENTAGE  OF  TOTAL 


EXHIBIT  15 


BAR  CHARTS  SHOWING  BUS  SCHEDULE  ADHERENCE  FOR  THE 
MINNEAPOLIS  URBAN  CORRIDOR  PROJECT 


FIGURE  D-5: 

I-35W  EXPRESS  SOUTHBOUND  - PM  SCHEDULE 
ADHERENCE  FOR  MTC  DOWNTOWN  SERVICE 
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VEHICLE  E 
(AUTO  AND 
TRUCK) 


EXHIBIT  17 

GRAPH  SHOWING  TRENDS  IN  BUS  UTILIZATION  FOR  THE 
SHIRLEY  HIGHWAY  EXPRESS - BUS - ON - FREEWAY  PROJECT 
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1969  1970  1971  1972  1973 


EXHIBIT  18 


TABLE  SHOWING  CORRIDOR  DEMOGRAPHIC  CHARACTERISTICS  FOR  THE 
SHIRLEY  HIGHWAY  EXPRESS-BUS-ON-FREEWAY  PROJECT 


TOTAL  CORRIDOR 

WITHIN  1 

MILE 

3ETWEEN  U2  MILES 

WEST  REMAINDER 

EAST  REMAINDER 

TOTAL 

PERCENT 

TOTAL 

PERCENT 

TOTAL 

PERCENT 

TOTAL 

PERCENT 

TOTAL 

PERCENT 

POPULATION 

Total 

496,470 

100 

132,945 

26 

93,861 

18 

149,607 

30 

120,057 

24 

Negro 

32,379 

7 

10,919 

8 

5,536 

5 

1,916 

2 

14,008 

12 

Number  Families 

167,564 

47,832 

32,079 

46,389 

41,264 

AREA 

Square  Miles 

Population  Density  per  Sq.  Mile 

152.6 

3,253 

42.6 

3,120 

30.3 

3,097 

49.5 

3,022 

30.1 

3,988 

YEAR  MOVED  INTO  HOUSING 

1968-1970 

73,871 

47 

24,725 

56 

13,392 

45 

19,668 

45 

16,086 

42 

1965-1967 

35,147 

22 

9,786 

22 

6,687 

22 

10,178 

23 

8,496 

22 

1960-1964 

24,117 

15 

5,575 

13 

4,320 

14 

7,933 

18 

6,287 

16 

1950-1959 

17,922 

11 

3,378 

7 

4,201 

14 

5,171 

12 

5,172 

14 

1949  or  earlier 

5,764 

4 

1,011 

2 

1,304 

4 

1,145 

2 

2,304 

6 

CLASS  OF  WORKER 

Private 

110,666 

52 

30,282 

51 

21,621 

53 

30,926 

50 

27,837 

55 

Government 

78,708 

37 

23,260 

39 

15,038 

37 

23,139 

38 

17,271 

34 

Local  Government 

16,372 

7 

4,370 

7 

2,721 

7 

5,264 

9 

4,017 

8 

Self-employed 

6,788 

3 

1,417 

2 

1,189 

3 

2,340 

3 

1.842 

3 

Total 

212,534 

100 

59,329 

28 

40,569 

19 

61,669 

29 

50,967 

24 

AUTOS  AVAILABLE 

1 

74,497 

44 

25,000 

53 

14,327 

44 

17,569 

37 

17,601 

41 

2 

60,004 

36 

14,488 

30 

10,963 

34 

21,261 

46 

13,292 

32 

3 or  more 

9,680 

5 

1,947 

4 

1,851 

6 

3,607 

7 

2,275 

5 

Total  (Autos) 

Average  (Autos/family) 
None 

223,545 

1.34 

25,179 

15 

60,617 

1.27 

6,291 

13 

39,955 

1.25 

4,518 

14 

10,912 

1.53 

4,815 

10 

51,010 

1.24 

9,555 

22 

MEANS  TRANSPORTATION  TO  WORK 

Driver 

147,958 

69 

41,186 

68 

27,647 

66 

44,972 

73 

34,153 

67 

Passenger 

30,186 

14 

8,763 

14 

6,155 

15 

8,164 

13 

7,104 

14 

Total  Autos 
Bus 

178,144 

21,906 

10 

49,949 

6,633 

11 

33,802 

5,096 

12 

53,136 

4,448 

7 

41,257 

5,729 

11 

Walked  to  work 

7,965 

4 

2,070 

3 

1,334 

3 

2,079 

3 

2,482 

5 

Worked  at  home 

3,352 

2 

881 

1 

622 

1 

1,059 

2 

790 

1 

Other 

4,107 

2 

1,092 

2 

844 

2 

1,113 

2 

1,018 

2 

WORK  PLACE 

D.C. -Central  Business  District 

20,095 

9 

S.973 

10 

3,967 

10 

5,920 

9 

4,235 

8 

D.C.  Remainder 

38,259 

18 

11,536 

19 

8,079 

19 

10,331 

17 

8,313 

16 

Arlington 

40,114 

19 

11,569 

19 

8,844 

21 

13,168 

21 

6,533 

13 

Virginia  Remainder 

88,847 

41 

23,885 

39 

15,935 

38 

24,624 

40 

24,403 

48 

Other 

28,241 

13 

7,688 

13 

4,924 

12 

7,812 

13 

7,817 

15 

Total 

215,556 

100 

60,651 

28 

41,749 

19 

61,855 

29 

51,301 

24 

a 

Refer  to  Figure  2 in  paragraph  2.3.3. 

Source:  U.S.  Department  of  Commerce,.  Bureau  of  the  Census,  Census  of  Population  and  Housing: 
1970  Census  Tracts  PHC  (l)-226,  Washington,  D.C.-Md.-Va.  SMSA,  May  1972. 
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EXHIBIT  19 

TABLE  SHOWING  PROJECT  OPERATING  STATISTICS  FOR  THE 
SHIRLEY  HIGHWAY  EXPRESS - BUS  - ON- FREEWAY  PROJECT 
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EXHIBIT  20 


TABLE  SHOWING  DEMOGRAPHIC  CHARACTERISTICS  OF  AUTO  USERS  FOR 
THE  MINNEAPOLIS  URBAN  CORRIDOR  PROJECT 

TABLE  B-13: 

DEMOGRAPHIC  CHARACTERISTICS  OF 
AUTO  USERS 


Arterial  Auto  User 

1 

-35W  Auto  User 

Characteristics 

PHI 

PH2 

PH3 

PHI 

PH2 

PH3 

% 

% 

% 

% 

% 

% 

Sex 

female 

48 

49 

46 

36 

36 

31 

male 

52 

51 

54 

64 

64 

69 

Age 

under  21  yrs. 

5 

5 

5 

3 

4 

3 

21  39  yrs. 

52 

51 

53 

55 

56 

56 

40  - 64  yrs. 

39 

40 

38 

40 

39 

40 

65+  yrs. 

4 

4 

4 

2 

1 

1 

Income 

under  $4999 

6 

5 

4 

2 

1 

1 

$5000  - 9999 

25 

26 

22 

15 

13 

9 

10000  - 14999 

30 

27 

22 

24 

20 

17 

15000  - 29999 

28 

33 

39 

44 

49 

53 

30000+ 

11 

9 

13 

15 

17 

20 

No.  of  Autos 

0 

4 

4 

3 

2 

1 

0 

1 

55 

51 

48 

37 

36 

33 

2 

33 

36 

40 

51 

51 

54 

3+ 

8 

9 

9 

10 

12 

13 

Length  of  Address 

under  11  mos. 

21 

21 

21 

18 

17 

17 

1 - 2 yrs. 

23 

22 

23 

21 

23 

24 

3 - 4 yrs. 

12 

10 

11 

15 

13 

13 

5-10  yrs. 

16 

19 

18 

22 

22 

21 

11  yrs.  + 

28 

28 

27 

24 

25 

25 
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TABLE  SHOWING  HIGHWAY  TRAVEL  TIME  DISTRIBUTIONS  FOR  THE  MINNEAPOLIS  URBAN  CORRIDOR  PROJECT 
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Significant  or  Not  Significant 
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CHAPTER  V 


RECOMMENDED  CONTENT  AND  ORGANIZATION  OF  REPORTS 


At  various  stages  in  the  evaluation  process,  the 
contractor  for  each  SMD  demonstration  project  is  responsible 
for  submitting  specific  reports  to  TSC.  These  reports  are 
an  Evaluation  Plan  submitted  prior  to  conducting  the 
evaluation.  Evaluation  Progress  Reports  submitted  monthly 
throughout  the  project.  Annual  Project  Status  Summaries 
submitted  yearly  throughout  the  project.  Interim  Evaluation 
Reports  submitted  periodically  throughout  the  project  (about 
once  a year),  and  a Final  Evaluation  Report  submitted  at  the 
conclusion  of  the  project.  The  grantee  for  each 
demonstration  project  is  currently  responsible  for 
submitting  quarterly  progress  reports  on  project  status.  As 
appropriate,  information  in  these  reports  will  be  included 
in  the  contractor’s  Monthly  Evaluation  Progress  Reports. 

This  chapter  presents  recommendations  on  content  and 
organization  which  will  guide  the  contractor  in  the 
preparation  of  these  reports.  The  suggested  content  and 
organization  for  the  grantee’s  quarterly  progress  reports  is 
also  presented. 


A.  EVALUATION  PLAN 

The  Evaluation  Plan  is  written  by  the  contractor  to 
explain,  in  detail,  how  the  evaluation  of  the  particular 
demonstration  project  will  be  performed.  The  following  is  a 
summary  of  the  suggested  content  and  organization  format  for 
the  Evaluation  Plan. 

(1)  Overview  of  the  demonstration  project 

• Planned  project  innovations 

• Demonstration  site 

• SMD  Program  objectives  addressed 

• Other  relevant  project  objectives/issues 
addressed 

• Project  history  (events  or  studies  leading 
up  to  demonstration) 
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• Project  schedule 

• Project  funding  (total  demonstration  costs 
by  source  of  funding,  demonstration  capital 
costs  by  innovation) 

• Project  grant  recipient/operating  agency. 

(2)  Description  of  the  demonstration  evaluation 

• Overview  of  basic  evaluation  design,  constraints 
affecting  development  of  the  Evaluation  Plan 

• Timing  of  evaluation  stages  as  related  to 
project  implementation  schedule 

• Site  data  collection  plans  and  sources 

• Quantitative  measures,  qualitative  measures, 
and/or  information  to  be  collected  in 
connection  with  each  project  objective  and  issue 

• Proposed  data  collection/derivation  and  analysis 
technique  for  each  measure 

• Schedule  of  data  collection  activities  associated 
with  the  evaluation,  and  identification  of  which 
organization  (contractor,  grantee,  other  local 
organization)  is  to  perform  each  activity. 
Schedule  should  indicate  submittal  dates  for  any 
Interim  Evaluation  Reports  and  the  Final 
Evaluation  Report. 

(3)  Technical  management  and  cost  information 

• Estimate  of  contractor  person-hours  by  type  of 
direct  labor  (e.g.,  project  manager,  junior 
analyst),  task  (i.e. , evaluation  plan  preparation 
and  updating,  management  and  coordination  of 
overall  evaluation,  data  collection/monitoring  of 
data  collection,  data  reduction,  data  analysis, 
and  report  preparation) , and  evaluation  stage. 

• Estimate  of  contractor  direct  costs  by  category 
of  cost  (travel,  computer,  etc.)  , task,  and 
evaluation  stage. 

• Estimate  of  total  contractor  evaluation  costs 

(Note:  detailed  cost  estimate  showing  hourly 

wage  rates,  overhead,  and  general  and  adminis- 
trative expenses  should  be  submitted  along 
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with  the  Plan  under  separate  cover.) 

• Estimate  of  person-hours  and  costs  for  data 
collection  to  be  performed  by  other  organiza- 
tions (by  activity  if  possible)  . 

To  facilitate  the  incorporation  of  modifications,  the 
Evaluation  Plan  should  be  submitted  in  looseleaf  form.  As 
modifications  are  made,  each  page  will  have  the  date  of 
modification  indicated.  Modifications  may  result  from  the 
initial  review  of  the  Plan  by  TSC,  UMTA,  and  the  grantee  or 
they  may  occur  during  the  evaluation  implementation  phase. 
As  an  example  of  the  latter  situation,  examination  of 
interim  findings  may  reveal  certain  gaps  or  redundancies  in 
the  originally  planned  data  collection  program.  Other 
reasons  for  modifying  the  Evaluation  Plan  during  the 
implementation  phase  might  be  operational  changes  in  the 
project,  unanticipated  developments  at  the  site,  or 
discovery  of  an  improved  evaluation  procedure.  The 
mechanism  for  obtaining  TSC  approval  to  modify  the 
Evaluation  Plan  procedures  is  described  below  under  Monthly 
Evaluation  Progress  Reports. 


B.  MONTHLY  EVALUATION  PROGRESS  REPORTS 

The  Monthly  Evaluation  Progress  Reports  are  written  by 
the  contractor  to  keep  TSC  and  UMTA  abreast  of  the  status  of 
the  demonstration  project  evaluation  the  contractor  is 
performing.  These  reports  are  intended  to  be  as  brief  as 
possible.  The  following  is  a summary  of  the  suggested 
minimum  content  and  organization  for  the  Monthly  Evaluation 
Progress  Report.* 


* Prior  to  the  start  of  the  project  evaluation,  when  the 
contractor  is  getting  familiarized  with  the  project,  it  is 
recommended  that  the  contractor  write  monthly  Progress 
Letters  describing  initial  contacts  with  the  project. 
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(1)  Review  of  evaluation  activities  during  the  past 
month.  Evaluation-related  problems  encountered 
and  actions  taken  to  rectify  them.  Narrative 
highlights  of  project-  and/or  evaluation- related 
exogenous  factors  and  other  events  which  appear  to 
be  significant  and  might  influence  the  evaluation 
of  the  project. 

(2)  Status  of  data  collection  and  analysis  activities 
that  have  taken  place  in  the  past  month  (performed 
by  both  grantee  and  evaluator) . Any  contractor 
documentation  on  preliminary  results  which  have 
been  generated  in  this  area  could  be  appended  to 
the  Progress  Reports  or  could  be  submitted 
separately  as  special  technical  memoranda. 

(3)  An  indication  of  whether  the  evaluation  is 
proceeding  according  to  schedule,  and,  if  not, 
reasons  for  the  departure.  A brief  discussion  of 
anticipated  activities  to  be  covered  during  the 
succeeding  report  period.  Forthcoming  Interim 
Reports,  if  any. 

(4)  Comparisons  of  cumulative  budgeted  to  actual 
expenditures.  Estimate  of  costs  to  complete 
evaluation  tasks. 

(5)  Recommendations  for  changes,  if  any,  to  the 
Evaluation  Plan,  and  the  reasons  such  changes  are 
recommended.  (TSC  concurrence  is  needed  before 
any  changes  to  the  Plan  can  be  made.) 


C.  ANNUAL  PROJECT  STATUS  SUMMARIES 

Annual  Project  Status  Summaries  are  written  by  the 
contractor  for  the  sole  purpose  of  furnishing  TSC  with  a 
"snapshot"  of  the  demonstration  project  at  the  specific 
point  in  time  when  the  TSC  annual  report  to  UMTA  is  written. 
It  is  envisioned  that  this  report  should  be  easily  prepared 
from  the  Evaluation  Plan  and  the  various  monthly  Progress 
Reports,  special  technical  memoranda,  and  Interim  Reports. 
The  suggested  content  and  organization  for  Annual  Project 
Status  Summaries  are  presented  below. 

(1)  Overview  of  the  project  describing  the  site,  the 

innovations,  the  SMD  Program  objectives  addressed, 
and  other  relevant  project  objectives  and  issues 
addressed. 
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(2) 


Overview  of  the  past  history,  present  operational 
status,  and  duration  of  the  project. 

(3)  Findings  to  date  in  terms  of  the  SMD  Program 
objectives  and  other  relevant  project  objectives/ 
issues. 

(4)  Implications  of  the  findings  relative  to  national 
significance  and  to  transferability  to  other 
locales. 

The  suggested  length  of  these  Status  Summaries  is  15  to 
20  pages,  with  charts  and  photographs  included  to  highlight 
the  narrative  descriptions.  Explanations  of  analyses 
performed  and  analysis  methodologies  used  are  not  needed. 


D.  INTERIM  EVALUATION  REPORTS 

Interim  Evaluation  Reports  are  written  periodically  by 
the  contractor  to  present  interim  findings  relative  to  all 
or  some  of  the  demonstration  project  objectives  and  to 
evaluate  those  aspects  of  the  project  where  it  is  applicable 
to  do  so.  Although  submitted  to  TSC,  they  are  also  meant 
for  dissemination  to  a technical  audience. 

Interim  reports  should  be  written  every  9 to  14  months. 
If  the  evaluation  process  is  divided  into  distinct  stages 
whose  durations  fall  roughly  within  these  time  frames,  then 
interim  reports  should  be  written  at  the  end  of  each  stage. 
If  the  stages  are  shorter,  such  that  two  or  three  stages  are 
completed  within  the  9 to  14  month  time  frame,  then  interim 
reports  should  be  written  at  the  end  of  these  break  points. 
Otherwise,  interim  reports  should  be  written  annually, 
except  that  no  interim  report  is  needed  at  the  end  of  the 
demonstration  project.  The  suggested  content  and  format  for 
Interim  Evaluation  Reports  are  presented  below. 

(1)  Executive  Summary  of  the  Report 

(Should  be  capable  of  standing  on  its  own  and 
being  published  separately.) 

(2)  Project  Overview 

Description  of  the  project  innovations,  the 
SMD  Program  objectives  addressed,  and  other 
relevant  project  objectives/issues.  Brief 
history  of  the  operation  of  the  project  over 
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its  life,  the  current  status  of  the  operation 
(at  the  end  of  this  report  period) , and 
highlights  of  pro ject- related  exogenous 
factors  and  activities  that  may  be 
significant  enough  to  influence  the  project. 

(3)  Site  Overview 

Description  of  the  site,  presentation  of 
pertinent  site  data,  and  highlights  of  site- 
related  exogenous  factors  that  may  be 
significant  enough  to  influence  the  project. 

(4)  Evaluation  Overview 

Description  of  the  basic  evaluation 
procedure,  the  timing  of  the  evaluation 
stages,  and  which  stages  are  being  examined 
in  the  report. 

(5)  Project  Results 

Assessment  of  the  project  in  terms  of  its 
attainment  of  relevant  SMD  Program  objectives 
and  other  (local  and/or  national)  project 
objectives;  insight  into  project  issues 
associated  with  operational  feasibility  and 
characteristics  of  the  demonstrated 
innovation.  (For  some  objectives  and  issues, 
no  concrete  statements  can  be  made  until  the 
project  is  concluded.)  Relevant  data  is 
analyzed  and  presented  in  the  forms  of 
charts,  graphs,  and/or  narrative.  (There 
will  generally  be  a need  to  present  data  from 
earlier  interim  reports.) 

(6)  Implications  Regarding  Transferability 

• Assessment  of  the  influence  of  si te- specif ic 
characteristics  and  exogenous  factors  on  the 
direction  of  the  demonstration. 

• Lessons  learned,  to  date,  based  on  practical 
experience,  relative  to  the  implementation 
and  operations  of  the  demonstrated 
innovations. 
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(7)  Appendix 


More  detailed  tables,  charts,  etc.,  on 
project  results. 


E.  FINAL  EVALUATION  REPORT 

The  Final  Evaluation  Report  is  structured  by  the 
contractor  early  in  the  demonstration  project  and  completed 
at  the  conclusion  of  the  project.  Its  purpose  is  to 
synthesize  the  findings  relative  to  each  of  the 
demonstration  project  SMD  Program  objectives  and  other 
relevant  objectives/issues  into  an  evaluation  of  the  overall 
project.  Although  submitted  to  TSC,  it  is  also  meant  for 
dissemination  to  a technical  audience.  The  suggested 
content  and  organization  for  the  Final  Evaluation  Report  are 
given  below. 

(1)  Executive  Summary  of  the  Report 

(Should  be  capable  of  standing  on  its  own  and 
being  published  separately.) 

(2)  Project  Overview 

Description  of  project  innovations,  the  SMD 
Program  objectives  addressed,  and  other 
relevant  project  objectives/issues.  Brief 
overview  of  the  operation  of  the  project  over 
its  life,  and  highlights  of  project-related 
exogenous  factors  and  other  events  that  have 
been  significant  enough  to  influence  the 
project. 

(3)  Site  Overview 

Description  of  the  site,  presentation  of 
pertinent  site  data,  and  highlights  of  site- 
related  exogenous  factors  that  may  have  been 
significant  enough  to  influence  the  project. 

(4)  Evaluation  Overview 

Description  of  the  basic  evaluation  procedure 
and  the  timing  of  evaluation  stages. 
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(5)  Project  Results 

Assessment  of  the  project  in  terms  of  its 
attainment  of  relevant  SMD  Program  objectives 
and  other  (local  and/or  national)  project 
objectives;  insight  into  project  issues 
associated  with  operational  feasibility  and 
characteristics  of  the  demonstrated 
innovation.  Relevant  data  is  analyzed  and 
presented  in  the  forms  of  charts,  graphs, 
and/or  narrative. 

(6)  Implications  Regarding  Transferability 

• Assessment  of  the  influence  of  site-specific 
characteristics  and  exogenous  factors  on  the 
outcome  of  the  demonstration. 

• Lessons  learned,  based  on  practical 

experience,  relative  to  the  implementation 
and  operations  of  the  demonstrated 
innovations.  Can  include  suggestions  for 
project  modifications  in  the  demonstration 
site  or  for  future  applications  in  other 
locales. 

(7)  Appendices 

• Project  costs 

• Data  Collection:  Site  data,  quantitative 

measures,  and  qualitative  measures  collected. 

• Assessment  of  evaluation  procedures  employed 

— e.g.,  effectiveness  of  particular  survey 
approaches  used,  cost/accuracy  of  innovative 
data  collection  techniques. 


F.  QUARTERLY  PROJECT  PROGRESS  REPORTS 

The  Quarterly  Project  Progress  Reports  are  written  by 
the  grantee  to  keep  UMTA/TSC  abreast  of  the  status  of  the 
demonstration  project  implementation  for  which  the  grantee 
is  responsible.  These  reports  can  serve  as  useful  input  to 
the  contractor's  work.  The  following  is  a summary  of  the 
suggested  minimum  content  and  organization  for  the  Quarterly 
Project  Progress  Reports. 
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(1)  Review  of  demonstration  project  activities  during 
the  past  quarter.  Project-related  problems 
encountered  and  actions  taken  to  rectify  them. 
Narrative  highlights  of  exogenous  factors  and 
other  events  which  appear  to  be  significant  and 
might  influence  the  project.  An  indication  of 
whether  the  project  is  proceeding  according  to 
schedule  and,  if  not,  the  reasons. 

(2)  Status  of  planned  data  collection  activities. 

(3)  Comparisons  of  budgeted  to  actual  expenditures. 
Estimate  of  costs  to  complete  project. 

(4)  Recommendations  for  changes,  if  any,  to  the 
conduct  of  the  project,  and  the  reasons  such 
changes  are  recommended. 
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APPENDIX  A 


SURVEY  EXECUTION  AND  DESIGN 


1 . INTRODUCTION 

It  is  anticipated  that  the  evaluation  of  every  SMD 
project  will  require  data  that  can  be  obtained  only  from 
surveys,  and  will  therefore' require  some  form  of  survey  data 
collection.  Amonq  the  possible  survey  respondents  are  SMD 
service  users,  auto  users,  service  area  residents  who  do  not 
use  transit,  and  transit  company  personnel.  Typical  survey 
objectives  miqht  include  determininq  user  and  non-user 
characteristics,  attitudes  toward  transit  service,  and  past 
and  present  travel  behavior;  measuring  modal  shift;  and 
assessing  the  experience  of  transit  officials  with  regard  to 
implementing  a new  technique.  Although  the  specific 
contexts  in  which  the  surveys  are  conducted  may  differ, 
there  is  still  a need  for  consistency  of  procedure  in  survey 
design  and  data  collection  to  insure  comparability  of 
results. 

In  surveys,  the  researcher  is  collecting  data  from  real 
life  situations,  which  means  that  many  unanticipated, 
spontaneous,  and  unusual  situations  will  arise.  To 
compensate  for  the  survey  researcher's  lack  of  control  of 
the  experimental  situation,  the  need  for  consistency  and  the 
establishment  of  general  policies  or  guidelines  to  handle  a 
great  variety  of  possible  developments  is  most  important. 

This  appendix  contains  guidelines  for  use  in 
formulating  and  carrying  out  surveys.  It  discusses  how  to 
define  the  populations  to  be  sampled,  i.e.,  the  survey 
universes,  describes  how  to  select  samples  that  will  be 
representative  of  that  universe,  examines  techniques  for 
surveyinq  the  samples  selected,  presents  suggestions  as  to 
survey  content  and  format  (including  a list  of  standardized 
questions  to  serve  as  a basic  set  for  most  surveys) , and 
discusses  the  problem  of  non-response  bias. 

A separate  section  at  the  end  of  this  appendix  contains 
guidelines  for  conducting  interviews  with  transit  company 
personnel,  e.g.,  drivers,  management,  mechanics. 

It  should  be  stressed  that  this  appendix  presents  no 
hard  and  fast  rules  which  must  be  followed  by  each 
contractor.  It  merely  guides  the  contractor  in  designing 
and  executing  surveys.  In  determining  survey  methodology, 
the  contractor  should  consider  potential  alternatives  and 
give  the  rationale  for  decisions  made  in  terms  of  the  survey 


A-l 


objectives,  site  characteristics,  and  any  other  relevant 
factors  which  have  influenced  the  decision. 


2.  DEFINING  THE  SURVEY  UNIVERSE 

The  first  step  in  executing  surveys  is  to  define  the 
survey  universe,  i.e.,  the  groups  about  which  the  surveys 
are  seeking  knowledge.  It  is  apparent  that  knowledge  about 
project  service  users'  travel  behavior,  characteristics,  and 
attitudes  toward  transit  is  needed  in  an  evaluation  of 
project  service.  Moreover,  an  evaluation  of  project  service 
will  usually  not  be  complete  without  some  data  on  non-users, 
particularly  to  identify  who  they  are  and  why  they  do  not 
use  the  project  service.  Accordingly,  there  are  two  survey 
universes  which  will  be  relevant  for  SMD  projects:  users  of 

the  demonstration  project  service,  and  non-users  of  the 
project  service.  Users  are  defined  as  those  who  ride  the 
project  service  at  least  occasionally  but  still  on  a regular 
basis  (e.g.,  regularly  twice  a month).  Non-users  are 
defined  as  those  using  alternate  modes  (i.e.,  other  than 
demonstration  project  vehicles)  who  make  trips  that  could  be 
made  on  the  project  service.  In  transit  dependent 
demonstration  projects,  non -users  could  also  be  persons  who 
could  make  trips  using  the  demonstration  project  service  but 
do  not  make  these  trips  at  all. 

Occasionally,  there  will  be  a third  survey  universe  of 
interest,  the  general  population  of  the  region  in  which  a 
demonstration  is  being  implemented.  Attitudinal  surveys  of 
this  universe  will  be  used  to  obtain  a profile  of  the 
community  in  which  the  demonstration  project  service  is 
being  provided.  It  should  be  apparent  that  many  of  the 
questions  asked  users,  non-users,  and  the  general  population 
will  be  different. 

Definition  of  the  term  demonstration  project  service 
area  allows  a more  precise  definition  of  non-users  and  the 
general  population.  The  project  service  area  is  defined  as 
the  area  that  comprises  on  the  order  of  90  to  95  percent  of 
the  origins  and  destinations  of  the  users  of  the 
demonstration  project  service.  Since  non-users  are 
potential  users,  the  origins  and  destinations  of  non-users 
should  be  comparable  to  those  of  users.  Non-users  can  now 
be  defined  as  persons  not  using  the  demonstration  project 
service  who  make  trips  that  begin  in  the  origin  portion  of 
the  service  area  and  end  in  the  destination  portion  of  the 
service  area  at  the  same  times  as  users  make  these  trips. 

The  general  population  in  the  region  of  the  demonstration 
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project  can  now  be  defined  as  the  population  residing  within 
the  service  area. 

The  demonstration  project  service  area  is  usually  not 
well  defined  at  the  outset  of  the  project  and  must  initially 
be  estimated.  In  some  demonstration  projects,  specifically 
demand-responsive  projects,  the  origin  and  destination 
portions  of  the  service  area  are  given.  At  the  other 
extreme,  in  demonstration  projects  where  park-and-ride  is  a 
significant  access  mode,  it'  may  be  impossible  initially  to 
estimate  the  service  area  accurately.  A conservatively 
estimated  area  that  includes  all  possible  park-and-riders 
would  have  to  be  initially  defined  as  the  origin  portion  of 
the  project  service  area.  Once  survey  data  on  the  origins 
of  park-and-riders  is  obtained,  a more  accurate  estimate  of 
the  service  area  can  be  made,  and  non-users  can  then  be 
identified. 


3.  SAMPLING  THE  SURVEY  UNIVERSE 

The  next  step  in  executing  surveys  is  selecting  an 
appropriate  sample  for  surveying  users,  and,  where 
applicable,  selecting  appropriate  samples  for  surveying  non- 
users and  the  general  population. 

The  purpose  of  sampling  is  to  reduce  the  amount  of  data 
collection  required.  Rather  than  obtaining  information  from 
every  member  of  the  universe,  the  principles  of  sampling 
provide  ways  to  obtain  information  from  a very  small  portion 
of  the  universe.  Sampling  procedures  also  indicate  the 
accuracy  with  which  the  characteristics  of  the  universe  have 
been  represented. 

A key  assumption  in  sampling  is  that,  prior  to  drawing 
a sample,  the  complete  universe  has  been  identified. 
Therefore,  every  member  of  that  universe  has  a known 
probability  of  being  selected  for  inclusion  in  the  sample. 
The  quality,  or  representativeness,  of  any  sample  is 
directly  derived  from  the  completeness  of  the  identification 
of  all  members  of  the  designated  universe. 

For  these  reasons,  careful  definition  of  the  universe 
and  selection  of  a source  from  which  to  draw  a sample  is 
very  important.  If  the  listing  of  the  universe,  or  the 
sampling  source,  is  biased  through  failure  to  include 
affected  members,  whether  deliberate  or  random,  the  sample 
may  magnify  the  bias  and  may  not  represent  the  universe. 
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A sample  of  users  can  be  selected  from  among  those 
onboard  the  transit  vehicles  or  among  those  at  transit 
collection  points,  i.e.,  stations,  park-and-ride  lots  or 
transfer  points.  For  demonstration  projects  in  which  all 
users  are  registered,  e.g.,  demand  responsive  or 
subscription  service,  a sample  can  be  selected  from  among 
the  registration  lists.  For  projects  which  serve  specific 
employment  or  activity  centers,  e.g.,  handicapped  and 
elderly  service  or  subscription  service,  a sample  can  be 
selected  at  these  centers. 

Selecting  a sample  of  non-users  is  considerably  more 
involved  than  it  is  for  users.  While  the  user  group  is 
identifiable  (and  can  be  directly  sampled)  , the  non-user 
grouD  cannot  explicitly  be  identified  before  it  is  sampled.* 
A larger  group  must  first  be  sampled,  and  then  the  trip 
origins  and  destinations  of  the  survey  respondents** 
examined  in  order  to  identify  non-users  (i.e.,  those  whose 
trip  origins  and  destinations  are  within  the  project  service 
area) . A definition  of  the  project  service  area  (as 
previously  discussed)  is  a prerequisite  for  identifying  non- 
users. 

In  demonstration  projects  where  travel  by  users  and 
non-users  is  in  a specific  direction  through  a corridor,  as 
on  the  Shirley  Highway  project  in  Virginia  or  the  I-95/N.W. 
7th  Avenue  project  in  Miami,  Florida,  non-users, 
specifically  auto  users,  can  be  sampled  from  license  plate 
matches.  A screenline  is  selected  which  intercepts  the  main 
arterials  carrying  autos  between  the  origin  and  destination 
portions  of  the  project  service  area.  A sample  of  the 
license  plate  numbers  of  the  autos  crossing  the  screenline 
is  recorded  and  a list  of  names  and  addresses  of  the  owners 
of  these  autos  is  obtained  from  Department  of  Motor  Vehicle 
records.  This  list  (or  a subset  of  this  list)  constitutes  a 
sample  in  which  a large  percentage  are  project  service  non- 
users. Some  of  those  crossing  the  screenline  do  not  make 
trips  that  begin  and  end  in  the  project  service  area,  and 
are,  therefore,  not  non-users.  However,  the  entire  sample 


*There  may  be  SMD  projects  directed  at  carpooling.  In  this 
situation,  carpoolers  would  be  "users”  as  defined  in  this 
appendix.  However,  the  population  of  carpoolers  is  not 
explicitly  identifiable;  therefore,  it  must  be  sampled  by 
the  same  methods  used  for  non-users. 

**This  information  is  requested  in  the  survey. 
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must  be  surveyed  because  it  is  not  known  who  the  non-users 
are  until  the  trip  origins  and  destinations  of  all  those  in 
the  sample  who  completed  their  stirveys  are  examined.  In 
certain  very  specific  cases,  samples  can  be  selected 
directly  from  the  traffic  stream,  e.g.,  at  toll  booths,  at 
off-ramps,  or  from  among  carpoolers  assembling  at  parking 
lots. 


In  demonstration  projects  where  travel  by  users  and 
non -users  is  not  in  a speci-fic  direction  nor  through  a 
corridor,  the  non-user  universe  cannot  be  sampled  using  the 
above  methods.  In  such  cases,  a sample  mav  be  drawn  from 
households  in  the  origin  portion  of  the  particular  project's 
service  area.  Lists  of  households  from  which  to  select  a 
sample  could  be  obtained  from  utility  records,  insurance 
company  records,  census  block  statistics,  telephone  books,* 
property  tax  records,  etc.  Many  of  the  people  in  these 
households  do  not  make  trips  ending  in  the  destination 
portion  of  the  project  service  area,  and  are,  therefore,  not 
considered  non-users.  As  previously  discussed,  the  entire 
sample  must  still  be  surveyed  because  the  non-users  cannot 
be  identified  until  after  the  entire  sample  is  surveyed. 

If  the  preceding  method  is  used  for  obtaining  a sample 
of  non-users,  it  should  be  noted  that  the  households 
selected  constitute  a sample  in  which  a moderate  percentage 
of  the  people  are  users.  It  may  be  desirable  to  identify 
users  before  they  are  surveyed  (by  asking  all  those  sampled 
if  they  are  users)  in  order  to  ask  them  questions  pertaining 
to  their  use  of  the  project  service. 

In  all  samples  of  households,  an  attempt  is  made  in 
each  household  to  survey  only  that  individual  in  each 
household  who  makes  a trip  ending  in  or  near  the  destination 


* Where  the  telephone  book  is  used  as  the  sampling  source, 
there  is  considerable  danger  of  obtaining  a biased  sample. 
About  20%  of  the  population  nationally,  and  up  to  40%  in 
large  urban  areas,  choose  to  have  unlisted  telephones. 

Also,  poor  people  are  much  less  likely  to  have  telephones, 
as  are  residents  of  boarding  houses. 

Random  digit  dialing  not  only  poses  potential  bias  problems 
but  also  will  be  costly  because  business  and  non-residential 
phones  will  be  called  up  to  60%  of  the  time  in  urban  areas. 
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portion  of  the  project  service  area.*  More  than  one 
household  member  is  surveyed  only  when  more  than  one  makes 
this  type  of  trip. 

For  demonstration  projects  which  serve  specific 
employment  or  activity  centers,  e.g.,  handicapped  and 
elderly  service  or  subscription  service,  a sample  of  non- 
users does  not  have  to  be  drawn  from  among  households.  A 
sample  can  be  selected  from  among  people  at  these  centers 
which  would  include  non-users  (and  users  also) . If  users 
are  surveyed,  they  should  be  identified  before  they  answer 
any  questions  in  order  that  the  questions  asked  pertain  to 
their  use  of  the  project  service. 

Where  a sample  of  the  general  population  of  a region  is 
needed,  the  sample  will  always  be  selected  from  among  the 
households  in  the  project  service  area.  Again,  lists  of 
households  can  be  obtained  from  utility  records,  insurance 
company  records,  census  block  statistics,  etc. 

Regardless  of  the  methods  chosen  for  selecting  samples 
of  both  users  and  non-users  (and  possibly  of  the  general 
population) , every  effort  should  be  made  to  assure  that 
samples  selected  are  unbiased  and  large  enough  for  the 
desired  statistical  confidence.  Such  an  approach  involves 
estimating  the  percent  of  persons  surveyed  who  are  in  the 
universe  (i.e.,  who  make  applicable  trips  in  the  service 
area) , estimating  the  response  rate,  and  developing  a random 
selection  process  that  aims  at  the  desired  number  of 
samples. ** 

In  developing  a random  selection  process  to  sample 
users  onboard  vehicles,  examination  of  vehicle  operating 
schedules  and  recent  passenger  counts,  if  available,  will  be 
necessary  to  design  where  and  when  to  select  the  vehicles  on 
which  to  sample  users.  However,  the  following  sources  of 
bias  in  vehicle  operating  schedules  must  be  considered  when 
deciding  on  the  utility  of  a particular  schedule  for 
developing  a sampling  source:  (1)  unscheduled  vehicle  runs, 

most  likely  to  occur  during  peak  hours,  and  therefore  with 


*This  comment  is  also  applicable  to  surveys  that  are  sent  to 
registered  automobile  owners  whose  names  were  obtained  from 
license  plate  matches. 

**See  Appendix  B for  a discussion  of  sample  size 
determination. 
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high  passenger  loads;  (2)  schedule  delays,  breakdowns,  and 
accidents,  also  most  likely  to  occur  during  peak  hours  when 
there  are  high  load  factors;  (3)  the  occurrence  of  exogenous 
influences  on  ridership  in  the  interim,  such  as  a strike 
among  people  who  might  have  formerly  used  this  mode  of 
transportation,  the  opening  of  a new  shopping  center  or 
school  along  the  route,  or  unique  events  such  as  a concert. 
These  sampling  hazards  should  be  kept  in  mind  and  some 
attempt  should  be  made  to  build  corrections  into  the 
research  design  to  compensate,  such  as  oversampling  on 
certain  routes. 

In  many  situations,  developing  a random  selection 
process  that  obtains  the  desired  sample  size  simply  involves 
selecting  every  Ith  person  going  past  a given  point,  or 
every  Jth  person  on  a list  of  users  of  a given  system,  or 
every  Kth  person  on  a list  of  employees  at  a given  location, 
or  writing  down  the  license  plate  number  of  every  Lth  auto 
going  past  a given  point.  To  obtain  a random  sample  of  the 
households  in  the  origin  portion  of  a project  service  area, 
every  Mth  household  on  a list  of  all  of  the  households  in 
the  area  could  be  selected;  or  the  random  clustered 
household  sampling  method,  used  on  the  San  Bernardino 
Freeway  Busway  project,  could  be  used.  This  method  divides 
the  origin  portion  of  the  service  area  into  smaller  areas 
(usually  blocks)  of  approximately  equal  population  and 
randomly  chooses  a sample  of  the  resulting  clusters  in  which 
every  household  in  each  cluster  is  a part  of  the  sample. 

The  possibility  of  sampling  bias  occurring  through  use 
of  a particular  sampling  method  should  not  rule  out  its  use. 
That  sampling  method  may  be  very  appropriate  in  certain 
project  evaluations.  However,  where  little  can  be  done  to 
minimize  the  effect  of  bias,  other  sampling  methods  should 
be  considered. 

For  each  survey  required  for  a particular  demonstration 
evaluation,  the  contractor  must  carefully  describe  the 
universe  to  which  survey  research  findings  will  be 
generalized  and  identify  the  most  complete  enumeration  or 
sampling  source  available  for  that  universe.  Actual 
selection  of  a sampling  source  must  be  justified  in  terms  of 
its  complete  coverage  of  the  affected  universe  and  also  in 
light  of  the  survey  objectives. 


4.  TECHNIQUES  FOR  SURVEYING  THE  SAMPLES  SELECTED 

The  final  step  in  executing  a survey  is  determining 
what  techniques  are  applicable  for  surveying  the  samples 
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that  have  been  selected.  There  are  five  basic  techniques 
for  surveying  these  samples: 

(D  Self-administered  questionnaires  handed  out  by 

individuals  (e.g. , survey  takers,  bus  operators, 
personnel  at  employment  or  activity  centers)  and 
collected  by  individuals  (not  necessarily  the  same 
ones  as  handed  out  the  questionnaires) 

(2)  Self-administered  questionnaires  handed  out  by 
individuals  and  returned  by  mail 

(3)  Self-administered  questionnaires  given  out  by  mail 
and  returned  by  mail 

(4)  Face-to-face  interviews 

(5)  Telephone  interviews. 

A summary  of  the  applicable  techniques  to  be  used  with  each 
possible  sampling  mothod  is  shown  in  Exhibit  A- 1 . 

With  all  of  these  techniques,  the  greater  the  amount  of 
personal  contact  between  user  and  survey  takers,  the  higher 
the  response  rate  and  the  quality  and  detail  of  the 
responses.  However,  the  greater  the  amount  of  personal 
contact,  the  higher  the  cost.*  In  fact,  the  face-to-face 
interview  initiated  at  homes,  while  eliciting  the  highest 
response  rate,  is  generally  too  costly  to  be  considered  in 
the  evaluation  process.  It  should  only  be  used  in 
conjunction  with  the  random  clustered  household  sampling 
method,  where  the  number  of  personal  home  interviews  to  be 
conducted  is  small  and  covers  a small  area.  By 
significantly  decreasing  the  area  in  which  a given  size 
sample  lies,  thQ  cost  of  using  personal  home  interviews  is 
reduced . 

Where  a self-administered  questionnaire  is  used  to 
survey  a sample,  the  response  rate  will  inevitably  be  lower 
than  where  a face-to-face  or  telephone  interview  is  used. 

To  improve  the  response  rate  it  may  be  desirable  to  allow 
for  a wave  of  follow-up  procedures,  such  as  phone  calls  and 
postcard  follow-up. 


*In  choosing  a survey  technique,  careful  attention  should  be 
paid  to  costs  associated  with  the  data  processing  and 
analysis  of  survey  findings. 
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EXHIBIT  A- 1.  SUMMARY  OF  SURVEY  SAMPLING  METHODS  AND  APPLICABLE  SURVEY  TECHNIQUES 


Generally,  the  self-administered  questionnaire  is  the 
most  easily  conducted  and  most  cost  effective  survey 
technique.  Self-administered  questionnaries  initiated 
onboard  or  at  collection  points  are  most  widely  applicable. 
If  the  questionnaires  are  short  enouqh  to  be  completed  by 
all  users  while  they  are  onboard  and  there  are  few  standees, 
the  users  should  be  instructed  to  complete  the 
questionnaires  while  onboard  and  return  them  as  they  leave 
the  vehicle.  If  the  questionnaires  are  initiated  onboard 
and  the  number  of  vehicles  on  which  users  are  surveyed  is 
not  large,  consideration  should  be  given  to  stationing 
survey  takers  onboard  each  vehicle  to  hand  out  and  collect 
the  questionnaires,  give  instructions,  and  answer  any 
questions.  If  the  questionnaires  are  initiated  at 
collection  points  and  the  number  of  points  at  which  users 
exit  their  vehicles  is  small,  consideration  should  be  given 
to  stationing  survey  takers  at  the  exit  points  to  collect 
the  questionnaires.  The  additional  expense  incurred  with 
this  degree  of  personal  contact  generally  pays  off,  i.e. , 
the  response  rate  is  high  and  the  cost  per  completed  survey 
is  low. 

Where  self-administered  questionnaires  are  too  long  to 
be  completed  by  all  users  while  they  are  onboard  or  where 
there  are  many  standees,  questionnaires  that  are  to  be 
mailed  back  should  be  used.  The  response  rate  for  a mail 
back  questionnaire  will  be  considerably  lower  than  for  a 
questionnaire  completed  onboard.  This  should  be  kept  in 
mind  when  developing  the  sampling  techniques. 

When  questionnaires  are  sent  by  mail,  a cover  letter 
giving  instructions  and  explaining  the  purpose  of  the  survey 
should  accompany  each  questionnarie  as  should  a self 
addressed,  stamped  envelope  for  mailing  back  the  completed 
questionnaire.  It  would  also  be  advisable  to  send  out 
"follow-up"  letters  a few  days  after  the  questionnaires  are 
sent  out  as  a reminder  to  complete  the  questionnaires. 

There  are  situations  where  it  is  advantageous  to 
conduct  personal  interviews  of  users  onboard  vehicles  or  at 
employment  or  activity  centers  rather  than  to  have  these 
users  complete  self -administered  questionnaires.*  Where 
the  total  user  population  to  be  surveyed  is  small,  a high 
response  rate  may  be  needed  to  obtain  the  desired 


*When  surveying  users  at  collection  points,  there  generally 
is  not  enough  time  to  question  them  by  personal  interview. 
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statistical  confidence.  In  such  a situation,  a self- 
administered  questionnaire  may  not  obtain  a high  enough 
response  rate,  while  personal  interviews  of  users  onboard 
vehicles  would.  Where  there  may  be  considerable  misgivings 
about  answering  a self-administered  questionnaire,  as  on  a 
crowded  bus  or  train  in  some  parts  of  some  large  cities, 
personal  interviews  conducted  onboard  vehicles  may  be  the 
only  means  of  obtaining  an  acceptable  response  rate.  Where 
the  users  being  surveyed  are  asked  about  concepts  or 
behavior  that  are  somewhat  complex,  a personal  interview 
will  be  much  more  effective  than  a self-administered 
questionnaire  in  eliciting  usable  responses.  Handicapped 
and  elderly  users  may  have  difficulty  writing  and  it  may  be 
difficult  for  them  to  respond  to  a lengthy  self- 
admininstered  questionnaire.  In  such  a situation,  personal 
interviews  conducted  onboard  or  at  activity  centers  are 
superior  to  self-  administered  questionnaires. 

Where  samples  are  selected  from  service  reqi strati on 
lists,  users  can  be  sent  self -administered  questionnaires  by 
mail.  Where  it  seems  that  a very  low  response  rate  would  be 
obtained  with  the  mail  back  questionnaire,  or  where  a high 
response  rate  is  necessary,  the  telephone  interview  would  be 
superior.  Moreover,  sampling  bias  would  be  minimized 
because  all  of  the  users’  telephone  numbers  would  be  known 
from  the  registration  lists.* 

For  surveying  non-users,  no  single  technique  is  widely 
applicable.  Where  a sample  of  auto  users  crossing  a 
screenline  is  surveyed,  questionnaires  could  be  sent  to  the 
auto  drivers  by  mail  (from  license  plate  matches)  or  these 
same  auto  drivers  could  be  interviewed  by  telephone;  or  auto 
users  selected  directly  from  the  traffic  stream  could  be 
given  questionnaires  to  be  returned  by  mail.  On  the  Shirley 
Highway  project,  for  example,  where  autos  were  selected  by 
license  plate  matches,  auto  occupancy  was  recorded  along 
with  license  plate  number,  and  mail-back  surveys  were  mailed 
out  according  to  auto  occupancy.  Those  who  drove  alone  were 
mailed  one  form;  carpool  drivers  were  mailed  a set  of 


*It  should  be  noted,  however,  that  it  will  not  be  possible 
to  contact  all  the  persons  in  the  telephone  survey  sample 
within  the  survey  time  frame.  Those  not  contacted  may  be  a 
nonrandom  group,  with  the  result  that  those  who  are  actually 
interviewed  by  telephone  may  no  longer  be  representative  of 
the  universe.  Therefore,  great  care  must  be  exercised  when 
sampling  by  telephone  interview. 
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different  forms:  a carpool  driver  form  for  themselves,  and 
carpool  passenger  forms  to  be  given  to  those  who  rode  with 
them. 

On  the  Miami  project,  where  autos  were  also  selected  by 
license  plate  matches,  the  owners  of  the  observed  autos  were 
surveyed  by  telephone  interview.  No  carpool  passengers 
could  be  surveyed  in  this  fashion.  A particular  set  of 
carpool  passengers  were  surveyed  directly  from  the  traffic 
stream.  Many  carpoolers  assembled  at  a parking  lot 
designated  partly  for  that  function.  Before  each  carpool 
left  the  lot,  each  member  of  the  carpool  was  given  a self- 
administered  questionnaire  to  be  mailed  back. * 

Where  a sample  of  non-users  (and  users  also)  is 
surveyed  at  specific  employment  or  activity  centers,  those 
techniques  which  are  applicable  for  user  surveys  initiated 
onboard  or  at  collection  points  should  be  considered.  This, 
in  general,  means  that  self -administered  questionnaires 
should  be  used. 

Where  a sample  of  households  in  the  origin  portion  of 
the  project  service  area,  which  includes  non-users  (and 
users) , is  surveyed,  no  single  survey  technique  is  widely 
applicable.  Questionnaires  could  be  sent  to  those 
households  by  mail  to  be  returned  by  mail,  telephone  home 
interviews  could  be  conducted,  or  personal  home  interviews 
could  be  conducted  where  the  sample  is  selected  using  the 
random  clustered  sampling  method. 

To  further  assist  in  the  selection  of  appropriate 
survey  techniques,  a summary  of  the  recent  survey  experience 
of  three  of  the  contractors  performing  evaluations  of  SMD 
projects  is  presented  in  Exhibit  A-2.  The  information 
presented  can  provide  a basis  for  olanning  of  similar  types 
of  surveys  during  the  various  SMD  projects. 

The  data  in  Exhibit  A-2  is  classified  by  three  primary 
survey  types,  namely,  onboard  vehicle,  household,  and 
telephone.  For  each  set  of  information,  the  specific 
contractor  is  identified  so  that  organizations  desiring  more 
detailed  information  can  make  appropriate  contacts.  In 
addition,  the  following  items  are  included: 


*Some  carpool  drivers  might  have  been  surveyed  twice  if 
their  license  plates  had  been  recorded. 
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EXHIBIT  A-2.  RECENT  SURVEY  EXPERIENCE  OF  CONTRACTORS 
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EXHIBIT  A-2  (CONT) 
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his  cost  is  based  on  a household  survey  of  the  San  Bernardino  Freeway  Corridor  which  searched  out  computers 
to  the  downtown  area.  Of  the  6672  houses  contacted  (8008  were  approached)  only  14%  had  a commuter(s)  whom 
we  were  interested  in  interviewing  and  only  8.6%  (576)  of  the  contacts  resulted  in  valid,  usable  responses. 


(1)  Locations  - an  identification  of  the  areas  in 
which  the  specific  surveys  were  conducted. 

(2)  Questionnaire  types  - an  indication  of  the 
specific  methodology  employed  for  the  survey. 

(3)  Duration/Length  - an  indication  of  the  time 
required  to  conduct  the  survey,  or  related 
information. 

(4)  Survey  costs  - experience  information  relative  to 
various  costs  including  additional  detail,  as 
appropriate. 

(5)  Response  rates  - an  indication  of  experienced 
response  rates  based  upon  total  sample 
information. 

(6)  Comments  - general  comments  emanating  from  these 
specific  surveys.  In  some  instances,  these 
comments  suggest  potential  areas  of  concern. 

In  addition  to  the  information  highlighted  in  Exhibit 
A-2,  the  following  comments  are  in  order  relative  to  the 
overall  experience  of  the  three  contractors  in  conducting 
surveys : 

(1)  Response  rates  greater  than  90%  occur  where  face- 
to-face  contacts  are  made.  In  addition,  even  in 
the  case  where  searches  are  required,  response 
rates  tend  to  be  fairly  high  in  face-to-face 
contact.  Mail  surveys  in  general  tend  to  produce 
response  rates  on  the  order  of  50%. 

(2)  Individual  survey  costs  can  vary  dramatically 
depending  upon  the  type  of  survey,  the  sample 
size,  and  the  response  rates.  Indications  in 
Exhibit  A-2  identify  rates  ranging  from  850  per 
questionnaire  to  as  much  as  $6.50  per  interview. 

(3)  There  are  two  primary  approaches  with  respect  to 
staffing  survey  operations.  The  first  involves 
untrained  local  personnel,  usually  obtained 
through  university  contacts,  but  under  the 
complete  training  and  supervision  of  an  individual 
from  the  consulting  firm  who  is  experienced  in 
such  surveys.  The  other  alternative  is  to 
contract  directly  with  local  organizations,  either 
personnel  or  market  research  firms. 
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(4) 


In  using  a universe  which  consists  of  a listing  of 
individuals  (e.g.r  a subscription  listing)  it  is 
critical  to  determine  how  up-to-date  the  listing 
is  as  well  as  how  easily  the  sample  can  be 
selected. 

It  is  important  to  recognize  that  the  data  contained  in 
Exhibit  A-2  is  for  planning  purposes  only  and  should  not  be 
construed  as  absolute,  particularly  in  the  response  rates 
that  were  experienced. 

It  is  anticipated  that  TSC  will  set  up  a "Survey 
Notebook"  in  which  will  be  kept  a record  of  the  survey 
experience  of  the  contractors  during  their  performance  of 
SMD  evaluations.  In  order  for  TSC  to  maintain  this 
notebook,  it  is  hoped  that  for  each  survey  conducted  for  SMD 
projects,  the  contractor  will  supply  TSC  with  a copy  of  the 
survey  form,  information  on  universe  size,  sample  size, 
cost,  and  response  rate,  and  reasons  associated  with  non- 
response. 


5.  SURVEY  DESIGN  PRINCIPLES 

It  is  apparent  that  because  different  surveys  are 
directed  at  different  survey  universes  using  different 
sampling  sources  and  different  techniques,  surveys  will  vary 
in  content  and  length.  Nonetheless,  all  surveys  should  have 
the  same  basic  organization,  sequence,  and  wording  of 
standardized  questions.  This  section  presents  basic 
principles  on  survey  organization,  length,  question  sequence 
and  wording,  and  standardized  questions  that  should  be 
followed  in  designing  the  survey  instrument. 


a . Organization 

There  should  be  four  elements  in  all  surveys,  whether 
user  or  non-user.  They  are  in  order  of  their  appearance  in 
a survey: 

(1)  Introduction  - This  is  a brief  statement  of  the 
survey’s  purpose  and  potential  utility  and 
guarantees  the  respondent’s  anonymity.  It  will  be 
verbally  delivered  if  an  interview  technique  is 
selected,  or  will  be  printed  at  the  beginning  of  a 
self-administered  questionnaire. 
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(2)  Behavioral  and  Attitudinal  Measures  - These  refer 
to  the  set  of  questions  specifically  measuring  the 
survey's  objectives,  such  as  modal  shift, 
satisfaction  with  level  of  service,  etc. 

(3)  Social  and  Demographic  Measures  - These  are 
measures  of  the  respondent's  characteristics  which 
are  important  in  interpreting  responses  to 
behavioral  and  attitudinal  measures.  Transition 
to  this  section  of  a survey  needs  to  be  prefaced 
by  either  a verbal  or  written  explanation,  as 
appropriate,  such  as  "Now  we  need  to  know  a little 
about  you...." 

(4)  Closing  Statement  - This  is  a brief  expression  of 
thanks  to  the  respondent  for  participating,  with 
some  indication  of  the  importance  of  the  eventual 
utilization  of  his  responses,  and  a request  from 
the  respondent  for  any  additional  comments  or 
observations. 


b.  Length 

The  overall  length  of  the  survey  depends  on  the 
particular  objectives  of  the  survey  and  the  survey 
techniques  used  (see  Section  4 of  this  Appendix) . In 
general,  surveys  which  are  to  be  completed  onboard  transit 
vehicles  and  at  employment  and  activity  centers  should  be 
shorter  than  those  surveys  completed  at  home,  since  they  are 
being  administered  to  respondents  in  a less  comfortable  and 
relaxed  environment. 

Se lf-administered  questionnaires  which  are  handed  out 
should  be  limited  in  length  to  one  side  of  a sheet  of  paper 
or  a large  postcard.  Surveys  which  are  to  be  completed 
onboard  transit  vehicles  and  at  employment  and  activity 
centers  (whether  in  interview  or  self-administered  format) 
should  be  shorter  than  surveys  which  can  be  filled  out  at 
the  respondent's  convenience  and  returned  by  mail. 

Moreover,  they  should  be  short  enough  so  as  not  to  delay  the 
respondent  in  his  trip  or  current  activity. 

The  length  of  surveys  which  are  completed  in  the  home 
varies  depending  on  the  method  of  administration.  Telephone 
surveys  should  be  fairly  short,  since  it  is  difficult  to 
retain  the  respondent's  attention  for  any  longer  period 
given  the  impersonal  nature  of  the  contact.  Self- 
administered  mail-back  questionnaires  given  out  by  mail  can 
be  longer  than  self-administered  mail-back  questionnaires 
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handed  out  because  there  is  more  opportunity  to  enlist  the 
respondent’s  cooperation.  However,  mail-back  questionnaires 
given  out  by  mail  should  not  be  as  extensive  as  personal 
interviews  conducted  in  the  home,  since  again  the  personal 
contact  is  lacking  which  might  encourage  a longer 
attention/cooperation  span  on  the  part  of  the  respondent. 


c.  Question  Sequence  and  Wording 

There  are  several  general  principles  describing 
question  sequence  and  wording  that  apply  to  all  questions. 
First,  questions  should  be  arranged  logically  to  lead  the 
respondent  into  the  frame  of  reference  of  the  issue  under 
study.*  It  is  recommended,  following  the  introductory 
material,  to  begin  the  questionnaire  or  interview  schedule 
with  behavioral  or  attitudinal  measures  of  responses  to 
transportation  alternatives  because  these  relate  most 
closely  to  the  announced  purpose  of  the  data  collection 
effort.  Social  and  demographic  data  should  be  collected 
near  the  end  of  the  survey  instrument,  reserving  any 
questions  about  income  as  near  to  the  end  of  the  survey  as 
possible. ** 

Questions  should  be  as  short  as  possible  and  in  clear, 
concrete  language.  Visual  format  is  also  important.  In 
self-administered  questionnaires,  it  enhances  the 
respondent*  s likelihood  of  completing  the  form,  and  in 


*See  pages  26  ff  in  Federal  Highway  Administration  with 
Urban  Mass  Transportation  Administration,  Urban  Mass 
Transportation  Travel  Surveys,  for  an  extended  discussion  of 
the  basic  considerations  in  designing  surveys.  Two  very 
practical  descriptions  of  interviewing  and  coding  guidelines 
helpful  in  developing  format  are  contained  in:  Survey 
Research  Center,  Interviewer's  Manual,  Institute  for  Social 
Research,  University  of  Michigan,  May  1969  and  Survey 
Research  Center,  A Manual  for  Coders,  Institute  for  Social 
Research,  University  of  Michigan. 

^♦Measures  of  income  are  the  most  difficult  to  obtain 
accurately  and  arouse  the  greatest  resistance  in  the 
respondent.  Sometimes  a respondent  is  asked  to  point  to  an 
amount  on  a card  or  circle  an  approximate  amount  to  lessen 
the  resistance.  However,  these  items  arouse  such  resistance 
that  they  must  be  at  the  end  of  the  data  collection 
instrument  so  the  hostility  produced  will  not  destroy  the 
rest  of  the  data  collection. 
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interview  format  surveys,  it  makes  the  interviewer's  task 
faster  and  easier.  Questions  should  be  laid  out  in  a 
fashion  that  ensures  ease  of  coding  and  keypunching 
responses  and  appears  attractive  at  the  same  time.  Fill-in 
questions  should  be  avoided  where  possible,  because  they  are 
difficult  to  code.  Where  they  are  used,  responses  should  be 
anticipated  and  precoded  to  reduce  costs  and  enhance 
consistency.  Coding  blocks  can  be  left  at  one  side  of  the 
survey  form  and  the  field  editor  can  check  that  information 
is  transferred.  This  procedure  makes  the  survey  also 
function  as  a code  sheet. 

The  survey  should  be  checked  to  ensure  that  it  is  as 
parsimonious  and  logical  as  possible.  There  are  several 
ways  to  do  this.  First,  every  question  ought  to  be 
evaluated  to  ensure  that  it  contains  a measure  related  to 
one  of  the  specific  project  objectives.*  Second,  advance 
planning  of  the  data  analysis,  through  the  construction  of 
dummy  tables,  will  ensure  that  every  variable  measured 
contributes  to  the  eventual  data  analysis.  Finally, 
pretesting  of  the  survey  instrument  will  identify  any 
questions  which,  because  they  are  confusing  to  the 
respondent  or  of  limited  use  in  the  evaluation,  should  be 
changed  or  omitted.  Pretesting  has  even  more  far-reaching 
benefits.  It  will  uncover  any  procedural  problems  which  may 
arise  during  the  survey  process  and  reveal  any  problems 
which  are  particularly  characteristic  of  urban  areas,  such 
as  a sizable  number  of  functional  illiterates  or  foreign 
speaking  respondents  who  cannot  complete  a self-administered 
questionnaire  or  a systematic  refusal  to  participate  by  some 
sectors  of  the  population.  The  pretest  of  the  survey  form 
must  be  conducted  with  respondents  as  identical  to  the 
prooosed  survey  respondents  as  possible  without 
contaminating  the  sampling  source. 


*There  are  several  exceptions  to  this  guideline.  One  is 
the  deliberate  use  of  one  or  two  meaningless  questions  in 
order  to  lead  the  respondent  into  a particular  frame  of 
reference.  This  is  frequently  necessary  when  seeking 
information  on  embarrassing,  unusual,  highly  specific  or 
complicated  issues.  This  technique  will  increase  the 
validity  of  the  data  subsequently  collected.  A second 
exception  is  measuring  respondent's  opinions  of  service 
features  that  have  not  changed  as  part  of  a set  of  questions 
about  respondents'  reactions  to  improved  service  features. 
This  combination  of  questions  will  measure  if  a "halo 
effect"  exists  in  terms  of  respondents'  overall  positive 
evaluation  of  the  mode  when  only  several  aspects  have  been 
changed. 
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Finally,  all  survey  questions  should  be  checked  against 
the  provisions  of  the  Privacy  Act  of  1974  to  verify  that 
none  of  the  questions  violates  any  person's  right  to  privacy 
as  spelled  out  in  the  Act.  It  is  recommended  that  the 
contractors  familiarize  themselves  with  the  provisions  of 
rhe  Act. 


d.  St andardiz ed  Questions 

It  will  be  useful  to  ensure  that  the  data  collected  in 
different  evaluation  projects  is  consistent  in  format. 
Fostering  consistency  means  that  an  economical  amount  of 
data  will  yield  a maximum  amount  of  information.  Secondly, 
consistency  facilitates  comparisons  between  projects, 
generating  a more  universally  applicable  understanding  of 
the  responses  to  transit  innovations.  Finally,  and  most 
importantly,  developing  consistent  data  collection 
categories  based  on  the  U.S.  Census  will  mean  that  results 
of  any  survey  can  be  corrected  for  sampling  error  and 
potentially  extrapolated  to  any  other  area.  This  section 
discusses  standardized  formats  for  measuring  behavioral, 
attitudinal,  and  social/demographic  characteristics. 


( 1 ) Behavioral  Measures 


Selecting  questions  to  measure  travel  behavior  is  very 
much  influenced  by  the  objectives  of  a particular  survey. 
Some  general  suggestions  regarding  ways  to  collect  and  code 
such  information  to  increase  consistency  among  surveys  will 
be  described. 

The  following  measures  of  travel  behavior  are  most 
likely  to  be  asked  in  almost  every  survey:  transit  vehicle 
boarding  and  alighting  points  (user  surveys  only) , trip 
origin  and  destination  (all  described  in  terms  of 
addresses) , trip  purpose,  and  trip  start  and  end  times. 
Additional  frequently  collected  data  for  surveys  includes 
access  mode  to  transit  vehicle,  when  present  mode  was  first 
used  for  this  particular  trip,  former  mode  used  for  this 
particular  trip  (with  some  attempt  to  control  for  exogenous 
influences,  such  as  a residential  move) *,  reason  for 
switching  mode,  fare  (user  surveys  only) , tolls  and  parking 
cost  (non-user  surveys  only) , frequency  of  use,  access  time 


*The  1974  Shirley  Highway  bus  user  survey  (see  Exhibit  A- 3) 
attempts  this. 
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a-t  origin  and  destination  (user  surveys  only)  , availability 
of  mass  transit  alternatives,  back-up  mode,  and  number  of 
transfers  required  (user  surveys  only) . 

Exhibits  A-3  through  A-6  are  examples  of  user  surveys; 
Exhibits  A- 7 through  A- 9 are  examples  of  non-user  surveys. 
These  exhibits,  together  with  the  preceeding  discussion, 
indicate  the  possible  range  of  information  which  can  be 
collected  on  travel  behavior.  Clearly,  the  determination  of 
which  particular  items  to  i-nclude  in  a survey  depends  on  the 
survey  objective,  desired  survey  length,  and  circumstances 
under  which  the  survey  is  conducted.  Furthermore,  the 
specific  wording  of  the  questions  relating  to  travel 
behavior  depends  on  the  method  of  administering  the  survey 
and  the  overall  tone  of  the  survey  and  sequence  of 
questions. 

Exhibits  A- 10  through  A- 16  present  recommended  question 
formats  and  response  categories  for  the  measures  of  travel 
behavior  which  are  likely  to  be  included  in  most  user  and 
non-user  surveys.  These  recommendations  are  based  on  a 
review  and  evaluation  of  questions  asked  in  past  surveys 
(including  Census  Journey-to-Work)  and  are  directed  to  the 
five  basic  types  of  surveys  (See  Exhibit  A-1).  In  designing 
a survey  for  a particular  demonstration  project,  the 
contractor  should  follow  these  guidelines  to  the  extent 
consistent  with  the  scope  and  objectives  of  the  survey.  Any 
significant  deviations  from  the  recommendations, 
particularly  modifications  of  suggested  response  categories, 
should  be  explained  to  TSC  in  a memorandum  accompanying  the 
draft  survey  instrument. 


( 2)  Attitudinal  Measures 


Attitudinal  items  will  be  used  in  many  surveys  to 
measure  the  respondent' s evaluation  of  the  transit  service 
provided,  specifically  in  terms  of  such  service 
characteristics  as  reliability,  convenience,  attractiveness, 
and  safety  of  alternative  modes.  Attitudinal  questions  may 
also  be  used  to  determine  what  factors  have  influenced  a 
modal  change.  Construction  of  such  items  requires  careful 
design  and  will  lengthen  the  survey's  administration  time. 
Occasionally,  attitudinal  questions  may  be  used  to  obtain  a 
profile  of  the  community  in  which  the  transit  service  is 
being  provided.  An  entire  survey  would  then  be  designed 
explicitly  for  the  purpose  of  determining  the  opinions  of 
the  general  population  in  the  project  service  area  to  such 
things  as  the  role  of  government,  environmental  issues, 
adequacy  of  transportation  facilities,  and  desirability  of 
travel  by  alternate  mode. 
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EXHIBIT  A-3 


SHIRLEY  HIGHWAY  CORRIDOR  SELF-ADMINISTERED 

BUS  SURVEY 

NBS-760  OMB  No.  41-R2752 

(g.73)  Approval  Expire*  12-31-76 

U.S.  DEPARTMENT  OF  COMMERCE 
National  Bureau  of  Standards 


Shirley  Highway  Corridor  Bus  Commuter  Survey 

This  survey  is  sponsored  by  the  U.S.  Department  of  Commerce 
THE  FOLLOWING  QUESTIONS  CONCERN  THE  TRIP  YOU  ARE  MAKING  THIS  MORNING 
Please  Answer  All  Questions  and  Mail  - No  Stamp  Required 


1.  At  what  comer  (or  park-and-ride  lot)  did  you  board  this  bus? 


(specify  nearest  street  intersection)  8 

How  did  you  get  from  the  place  where  this  trip  began  to  the  place  where  you 
boarded  this  bus? 

□ walked  □ was  driven  by  another  person 

□ drove  car  and  parked 

□ other  (specify) 


□ 


3.  Where  did  this  trip  begin?  (Your  home  address  if  this  trip  started  at  home) 

Street  Address  12 

City  42  State  69  Zip  Code  7 

4 This  address  was  □ home  □ other  (specify) 

5.  Time  you  began  this  trip A.M.  (left  above  address)  1 I 1 1 

76  7S 

6.  What  was  the  final  destination  of  this  trip?  {Physical  address  of  your  work  place, 
if  work  trip) 

Street  address  or  building  name  12  4' 


lD 


7.  This  address  was  □ work  □ other  (specify) 

8 Time  this  trip  ended A.M.  (arrived  at  above  address) 


n 


76 


79 


When  you  made  this  trip  how  much  time  did  you  spend  driving  (or  being  driven) 
from  the  place  where  this  trip  began  to  the  place  where  you  left  your  auto? 

□ not  applicable,  or minutes 


10.  When  you  made  this  trip  how  much  time  did  you  spend  walking  to  and 
from  bus  stops  and  waiting  for  buses? minutes 


10 


1 1 . When  you  made  this  trip  how  many  times  did  you  change  (transfer)  buses?  | 

□ none,  or transfers  13 

12.  When  did  you  begin  to  regularly  use  this  bus  to  commute  from  |~ 

home  to  work?  □ not  applicable,  or month year  14  17 

PLEASE  CONTINUE  TO  QUESTION  13  ABOVE 


20. 


22 


□ 


13.  How  often  do  you  use  each  of  the  following  means  to  travel  from  home  to  work? 

a)  This  bus (0,1,2,etc.)  day  (s)  per  week 

b)  Driving  alone day(s)  per  week  L— -I ' 

c)  A carpool day(s)  per  week 

14.  Does  this  bus  arrive  at  your  boarding  bus  stop  later  than  the  scheduled  time?  |_ 

□ never  □ seldom  □ usually  □ always  21 

15.  Does  this  bus  arrive  at  your  destination  bus  stop  later  than  the  scheduled  time?  | 

□ never  □ seldom  □ usually  □ always 

16.  On  an  average  day  when  you  board  this  bus  do  you  find  a seat? 

□ never  □ seldom  □ usually  □ always  23 

17.  If  you  could  not  commute  from  home  to  work  by  means  of  this  bus  how  would 
you  usually  make  the  trip? 

□ would  be  unable  to  make  this  trip  I 1 

□ join  or  form  a carpool  □ use  another  bus 

□ drive  alone  □ other  (specify) 

18.  Before  you  began  using  this  bus  how  did  you  usually  commute  from  home  to  work? 

□ did  not  make  this  trip  (from  your  present  home  to  your  present  work  place), 
how  did  you  commute  prior  to  changing  your  place  of  residence  or  work? 

□ auto  □ bus  □ other  1 1 1 

□ drove  alone  1 | [ 

□ was  an  alternate  driver  in  a carpool  with other  person(s)  25  26 

□ drove  in  a carpool  with other  person(s)  (always  or  nearly  always  drove) 

□ was  a passenger  in  a carpool  with other  person(s)  (never  or  almost  never 

drove) 

□ used  another  bus  (specify  route) 


□ other  (specify) 

19.  If  prior  to  riding  this  bus  you  commuted  regularly  by  automobile 
(as  either  a driver  or  passenger),  why  did  you  switch  to  bus? 


If  prior  to  riding  this  bus  you  commuted  regularly  by  auto,  what  was  the  vehicle 
parking  cost?  ( Don't  divide  by  the  number  of  persons  sharing  the  parking  cost) 

□ not  applicable,  or  $ per  day. 

21 . During  the  past  two  years  has  this  bus  service  enabled  you  to: 

a)  dispose  of  a car  which  you  owned?  □ yes  □ no 

b)  avoid  buying  a car?  □ yes  □ no 

Did  this  bus  service  influence  the  choice  of  your  present  address? 

□ yes,  definitely  □ slightly  □ not  at  all 


29 


□ 


What  are  your  regular  working  hours?  □ no  regular  working  hours,  or_ 


_A.M.  to_ 


_P.M. 


□ don’t  drive 


When  was  the  last  time  you  changed  your  place  of  residence?  □ not  within  the  last  5 years,  or month year 

When  was  the  last  time  you  changed  your  physical  work  location?  □ not  within  the  last  5 years,  or month year 

Is  an  auto  available  for  you  to  REGULARLY  drive  alone  from  home  to  work? 

□ no  □ yes,  but  with  considerable  inconvenience  to  others  □ yes,  and  without  inconvenience  to  others 

How  many  automobiles  are  owned  or  operated  by  members  of  your  household?  □ none,  or auto(s) 

Which  of  the  following  attitudes  best  expresses  your  opinion  about  using  bus  as  a way  of  commuting  from  home  to  work? 

□ I am  generally  satisfied  with  using  bus  as  a long  range  solution  to  my  commuting  problems. 

□ I am  generally  satisfied  with  using  bus  as  a short  term  solution  to  my  commuting  problems  until  I can  use  metro  subway  service. 

□ I am  generally  dissatisfied  with  using  bus;  why 

□ Other  (specify) 


29.  Please  indicate  your:  Sex:  Ornate  □ female 

Age:  □ under  21  □ 21-39  □ 40-65  Do 

30.  What  is  the  combined  annual  income  of  all  members  of  your  household? 

□ $0-5,000  □$5,001-15,000  □ $15,001-30,000  □ above  $30,000 

31.  Any  comments? 


□ 

51 

□ 

52 

□ 


□ 


THANK  YOU  - PLEASE  SEAL  AND  MAIL 
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EXHIBIT  A-4 

MINNEAPOLIS  SELF -ADMINISTERED  AUTO  USER  SURVEY 


Survey  is  sponsored  by  U.S.  Dept,  of  Transportation 

Please  give  us  a few  minutes  and  we  will  try  to  give  you  a better  Transportation  system. 


Commissioner  of  Highways 


A 


Chairman,  Metrepnlifan  Transit  Commission 


CMB  0*1  R5537 


The  place  I came  from  was 


(Address  or  Street  Intersect  ion) 
The  place  I was  headed  for  was 


Home? 


.□  YES 

□ yes 


3.  I began  this  trip  at  about 


(Address,  Building  or  Street  I n ter  sect i onT~ 

A.M.;  and  arrived  at  my  destination  at  about 


□ HO 

□ NO 
A.M. 


4.  What  are  your  regular  working  hours? 

5,  When  I made  this  trip  I was: 


A.M. 


CD  the  driver  (with 


passengers ) 


□ alternate  driver  of  a car  pool 
(~D  regular  passenger  and  pay  $ 


pe  r 


Do  you  normally  use  this  car  during  the  business  day? 
What  is  the  vehicle  parking  cost?  per  


□ A.n. 

□ P-H. 


□ FREE 

□ YES  □ NO 
(day,  week,  nonth,  year) 


n f-!ee 


If  you  formerly  traveled  by  bus,  what  is  the  major  reason  you  switched  to  the  auto? 


0 cheaper 
Q safer 


0 less  travel  time 
0 more  dependable 


0 o t h e r_ 


9.  Could  you  have  used  a local  bus  to  make  this  trip? 

Byes,  but  chose  not  to  because  

no,  because  

0 don  1 1 know 

10.  Could  you  have  used  the  I-35W  Express  bus  service  for  this  trip? 

0yes,  but  chose  not  to  because 

Bno,  because  

don't  know 


(Specify) 

0 did  not  make  trip  by  b-;S 


11. 


12. 


If  one  transit  improvement  could  be  made,  which  of  the  following  would  induce  you  to  ride 
the  bus?  (Check  only  one ) 


Blov/er  fare 

More  frequent  service 
n Less  traveling  time 


0 More  comfortable  vehicle  0 More  extensive  service 
0 No  need  to  transfer  (less  walking) 

U n Other 

(Specify) 


How  would  you  rate  your  general  attitude  toward  traveling  to  work  by  bus? 

0 Very  positive  0 Positive  0 Negative  0Very  negative 


LET 


ID 


id 


□ 

ID 


□ 

□ 

□ 

□ 

□ 


Now,  just  a few  questions  for  statis tical  purposes : 

How  long  have  you  lived  at  your  present  address?  Years 


Months 


14.  How  many  autos  are  owned  or  operated  by  members  of  your  household? 

O None  □ One  0 Two  □ More  than  two  autos 

15.  Are  you:  0Male  CD  Female  What  is  your  age?  

16.  My  combined  annual  family  income  is: 

□ Under  $3,000  □ $5 ,000-$7 , 909  □ $1  0 , 000- $1  1 ,999.  □ $ 1 5 , 000- $ 1 9 , 999  □ $25  , 000-529  ,999 

□ $3,000-54,999  □$8,000-59,999  □ $ 1 2 ,000- $ 1 4 ,99$:  □ $20 .000-S24 ,999  □$30,000  or  more 


□ 


TH 


THANK  YOU  •Please  seal  and  mail 
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SOURCE: 


EXHIBIT  A- 5 

SEATTLE  BLUE  STREAK  SELF - ADMINI STERED  BUS  SURVEY 
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FHWA  and  UMTA , Urban  Mass  Transportation  Travel 
Surveys,  Exhibit  4-19. 
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EXHIBIT  A-  6 


WASHINGTON,  D.C.,  SELF-ADMINISTERED  POST  CARD 
BUS  SURVEY 


TO  BUS  RIDERS:  Please  help  t»  plan  your  future  buy  aid  ubaay  syitems  by  telling  us  ABOUT 
THE  TRIP  YOU  ARE  NOW  TAKING.  Drop  completed  cad  In  box  at  door  or  hand  to  driver.  If  you 
don't  have  a pencil  now,  the  card  can  be  mailed  tree. 


1. 

2. 

3. 


I got  on  this  bui  ot 


(NEAREST  STREET  CORNER)  (ClTr  OR  COMMUNITY) 

I hove  come  from:  Ohom*  Owork  Qthopping  Qichool 

Thit  ploce  I havo  com*  from  it  ot 

(ADDRESS  OR  NEAREST  CORNER) 


□ other 


(CITY  OR  COMMUNITY) 


4.  I am  getting  off  thit  but  at. 

5 I am  now  h*od*d  for: 

6.  Tbit  plac*  I am  headed  for  it  at . 

N?  465003 


(NEAREST  STREET  CORNER)  (CITY  OR  COMMUNITY) 

□ home  Qwork  Dthopping  Qtchool  Oother 


(ADDRESS  OR  NEAREST  CORNER) 


(CITY  OR  COMMUNITY) 

PLEASE  FILL  OUT  BOTH  SIDES 


BUDGET  BUREAU 
NO.  1)7-4001 

UNITED 

STATES 

GOVERNMENT 

NATIONAL 

CAPITAL 

TRANSPORTATION 

AGENCY 


PLEASE  FILL  OUT  BOTH  SIDES 

7.  How  did  you  get  to  thi t but: 

0Walked  0Drove  and  Parked  0C  or  Passenger 

0Bus  0Train  0Taxi  or  Other 

8.  How  will  you  get  from  thit  bus  to  where  you  are 
headed: 

□ Walk  OBu. 

(Rout*  Number  of  bus  rronsfarred  to) 

0Car  DToxi  or  Other 

9.  Check  the  number  of  cars  in  your  household: 

0None  01  Car  02  Cart  0More  than  two 

IF  YOU  TRANSFER  ON  THIS  TRIP  PLEASE  DO  NOT 
TAKE  ANOTHER  CARO 

TNAHK  YOU 


UNITED  STA'TES  GOVERNMENT 
NATIONAL  CAPITAL 
TRANSPORTATION  AGENCY 
WASHINGTON,  O.C.  20432 

OFFICIAL  BUSINESS 


NCTA  TRANSIT  SURVEY 
P.O.  BOX  9366 
WASHINGTON,  DC. 
20005 


POSTAGE 
AND  FEES 
PAID 
N.C.T.A. 


SOURCE: 


FHWA  and  UMTA , Urban  Mass  Transnortation  Travel 
Surveys,  Exhibit  4-3. 
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EXHIBIT  A-7 


SHIRLEY  HIGHWAY  CORRIDOR  SELF-ADMINISTERED 
CARPOOL  DRIVER  SURVEY 


OMB  No.  41-874087 
Approve!  Expires  12-31-74 
U S DEPARTMENT  OF  COMMERCE 
National  Buraau  of  Standards 


□ 


Shirtoy  Highway  Corridor  Carpootar  Survey 

Thii  survey  it  sponsored  by  the  U.S  Department  of  Commerce 
Ware  you  the  driver  in  a carpool  (that  ii,  an  automobile  carrying  2 or  more  periont,  including 

the  driver)  on  the  morning  of between  8 30  A.M.  end  9:00  A.M.? 

O yei,  pleaie  answer  the  following  questions  and  mail  survey  form  - No  Stamp  Required. 

□ no,  please  return  survey  form  without  answering  questions 


LISTED  BELOW  ARE  SEVERAL  FACTORS  WHICH  MIGHT  FIGURE  EITHER 
POSITIVELY  OR  NEGATIVELY  IN  A PERSON  S DECISION  TO  FORM  OR 
JOIN  A CARPOOL  HOW  IMPORTANT  WAS  EACH  OF  THESE  FACTORS  TO 
YOU  WHEN  YOU  FIRST  DECIDED  TO  USE  THE  CARPOOL  IN  WHICH  YOU 
WERE  RIDING  ON  THE  MORNING  CITEO  ABOVE? 


3 • Very  Important 
2 « Moderately  Important 
1 - Unimportant  or  Didn't  Consider  It 
0 - Not  Applicable 

rt  of  the  vehicle(s)  used  by  the  carpool  (e.g.,  leg  room,  air 
oning.  etc.) 

lion  in  overall  commuting  costs 

parking  privileges  provided  by  employer  for  cerpools 
terut.es  of  the  other  member(s)  of  the  carpool  (e.g.,  personality. 

i,  whether  person  is  a smoker  or  objects  to  smoking,  etc.) 
. Availability  of  Shirley  Highway  express  lanes  for  carpool  use 
. Lou  of  flexibility  in  working  hours 
. Availability  of  carpool  locator  services 

_ Additional  trip  time  resulting  from  passenger  pick  up  and  discharge 
. Reduced  use  of  an  auto  or  making  the  purchase  of  an  auto  unnecessary 

_ Concern  for  energy  and  air  pollution  problems 
_ Convenient  work  locetion(s)  of  the  other  member(s)  of  the  carpool 
. Reduciiqn  in  the  use  of  gasoline 

. Reduced  stress  end  frustration  in  commuting 
. Availability  of  good  bus  service  as  "back  up"  transportation 
. Other  factors  (specify)  


_ Special  par 
. Character:! 
punctualih 


What  intervening  stop(s)  did  you  make  t 
trip  began  end  your  final  destination,  oil 
discharge  carpool  passengers? 

□ none  □ restaurant 

□ babysitter  □ other(s)  (specify)_ 


□ 


PLEASE  CONTINUE  TO  QUESTION  4 ABOVE 


? (Your  home  at 


5.  This  address  was  □ home  □ other  (specify) 

6,  Time  you  began  this  trip A M.  (left  above  address) 


_□ 


7 What  was  the  final  destination  of  this  trip?  /Physical  address  of  your  work  p 


_□ 


(EXCLUDING  YOURSELF) person!. I 

1 1 When  you  made  this  trip  where  did  you  pick  up 
(EXCLUDING  YOURSELF) 

persons  were  picked  up  at  other  location!*) 

12.  When  you  made  this  trip,  how  many  persons  in  i 


□ not  applicable,  o 

□ on  the  street  □ in  a s 

□ in  a commercial  lot/garage  empli 

□ other  (specify) 

IS  What  is  the  vehicle  parking  cost?  I Don’t  c 


— r □ 

m 

Jr person(s)  | j 

jp? 

m 
In 


king  cost)  □ not  applic 


□ 


le  following  questions  (a-o)  refer  to  the 
•mber  of  this  carpool  please  check  here 
When  did  you  begin  to  regularly  use  th 

work? month year 

How  many  persons  usually  commute  f 

(EXCLUDING  YOURSELF) | 

How  often  are  you  the  driver  of  this  ci 


□ 


reserved  to  buses  ai 


erson(t) 


m 


□ alternately  drive 

d)  During  the  last  year  has  the  membership  of 
increased  for  any  reason? 

□ no 

□ yes.  (EXCLUDING  YOURSELF)  increas 

e)  What  motivated  the  increase  in  membership 

□ the  opening  of  Shirley  Highway  expreu  I 

□ an  employer  policy  providing  parking,  es 
some  minimum  membership 

□ carpool  locator  services 

□ the  gasoline  crisis 

□ other  (specify) 


□ □ 


f)  ! 


is  carpool (0,1.2,etc  ) day(s)  per  week 

iving  alone day  Is)  per  week 

s dsy(s)  per  week 

g)  How  many  regular  users  of  this  carpool  (EXCLUDING  YOURSElF) 

i.  members  of  your  household?  □ none,  or person(s) 

ii.  employed  at  your  place  of  work?  □ none,  or person(s) 

h)  Does  the  number  of  people  using  this  carpool  vary  from  day  to  day? 

□ stays  the  same  from  day  to  day  (except  for  sick  days  and  vacation! 

□ varies  somewhat  from  day  to  day 

□ varies  substantially  from  day  to  day 

i)  How  do  the  participants  in  this  carpool  share  the  vehicle  operating  an 

parking  costs?  (e.g..  share  driving,  pay  drivei  fixed  amount,  etc.) 


lid  a carpool  matching  service  auist  you  in  joining  or  forming 
lt/s  carpool?  Oyes  Ono 

I you  could  not  commute  from  home  to  work  by  means  of  this 
arpool  how  would  you  usually  make  the  trip? 

] would  be  unable  to  make  this  trip 
] join  or  form  another  carpool  □ use  bus 

] drive  alone  □ other  (specify) 

lefore  you  joined  this  carpool  how  did  you  usually  commute  fr< 

] did  not  make  this  trip  (from  your  present  home  to  your  presr 
work  place);  how  did  you  commute  prioi 
of  residence  or  work?  □ auto  □ bus 
D drove  alone,  which  roadway  was  used? 

□ Columbia  Pike  □ Shirley  Highway  □ Route  1 

□ Arlington  Blvd.  □ George  Washington  Parkway 

□ Other  (specify) 

] was  an  alternate  driver  in  a carpool  with other  person(s) 


□ 

□ 


m 


D drove  in  a carpool  with other  person(s)  (i 

D was  a passenger  ir 


arly 


m 

”□ 

m 


□ usee 

□ oth. 

n)  Before 

o)  Before 

□ did 

□ in  a 


cify  route) 


s from  home  to  work? minutes 

ou  joined  this  carpool  where  did  you 


in 


□ on  the  street 

□ in  a commercial  loi/garage 

□ other  (specify) 


□ 


PLEASE  CONTINUE  TO  PART  j OF  QUESTION  17  ABOVE  . 


Would  your  employer  provide  you  with  parking  if  you  were  not  participating  in  . 

What  are  your  regular  working  hours?  □ no  regular  working  hours,  or A.M 

When  was  the  last  time  you  changed  your  place  of  residence?  □ not  within  the  I 
When  was  the  last  time  you  changed  your  physical  work  location ? □ not  within 
Is  an  auto  available  for  you  to  REGULARLY  drive  alone  from  home  to  work? 


□ yes.  but  with  considerable  inconvenience  to  others  □ 
y automobiles  are  owned  or  operated  by  member 


s,  and  w 


of  the  following  attitudes  best  expresses  yi 
ti  generally  satisfied  with  carpooling  as  a la 
■n  generally  satisfied  with  carpooling  as  sth 


i household?  □ none,  or euto(s) 

carpooling  as  a way  of  commuting  from  homo  to  work? 
to  my  commuting  problems, 
to  my  commuting  problems  until 


□ I am  generally  d 

□ Other  (specify) 
Please  indicate  you 


stisfied  with  carpooling,  why  _ 


□ Male  □ Female 
What  is  the  oombined  annual  income  of  all  members  o' 


Age  □ under  21  0 21-39  0 4065  □ over  65 

ir  household?  □ $0-6.000  □ $5,001  16.000  □ $16,001-30,000  □ above  $30,000 


g 


□ 

□ 

□ 

m □ 


THANK  YOU  - PLEASE  SEAL  AND  MAIL 
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EXHIBIT  A-8 


SHIRLEY  HIGHWAY 
SOLO 


NBS-796 

(8-74) 


CORRIDOR  SELF-ADMINISTERED 
DRIVER  SURVEY 

OMB  No.  041-S74076 
Approval  Expire*  12-31-74 
U.S.  DEPARTMENT  OF  COMMERCE 
National  Bureau  of  Standards 


Shirley  Highway  Corridor  Single  Occupant  Auto  U»r  Survey 

This  survey  i*  sponsored  by  the  U S Department  of  Commerce 


□ 

8 


Did  you  make  a trip  by  driving  alone  in  an  automobile  on  the  morning  of between 

6:30  A M and  9 00  A.M.? 

O yes;  please  answer  the  following  questions  and  mail  survey  form  — No  stamp  required. 

□ no;  please  return  survey  form  without  answering  questions. 


What  intervening  stop(s)  did  you  make  between  the  place  where  this  trip 
began  and  your  final  destination? 

□ none  □ restaurant 

□ babysitter  □ other  Is)  (specify) 


□ 


When  you  made  this  tup.  where  did  you  park  your  auto? 

□ on  the  street  □ in  a commercial  lot/garage 

□ in  a space  provided  or  subsidized  by  your  employer 

□ other  (specify) 


□ 


3.  Where  did  this  trip  begin?  (Your  home  address  if  this  trip  started  at  home) 


10.  VAiat  is  the  vehicle  parking  cost?  (Don't  divide  by  the  number  of 

persons  sharing  the  parking  cost)  O not  applicable,  or  $ per  day 


city  42  Sun  99  Zip  Corn  TO 

This  address  was  □ home  □ other  (specify) 


Time  you  began  this  trip A.M,  (left  above  address) 

78 

What  was  the  final  destination  of  this  trip?  (Physic*/  address  of  your  work 
place,  if  work  trip) 


TO 


11. 

12. 

13. 


city  42  Sun  69 

This  address  was  □ work  □ other  (specify)  

Time  this  trip  ended A.M.  (arrived  at  above  address) 

PLEASE  CONTINUE  TO  QUESTION  9 ABOVE 


How  often  do  you  use  each  of  the  following  means 
work? 

a)  Driving  alone day (s)  per  week 

b)  Bus day(s)  per  week 

c)  A car  poo  I day(s)  per  week 


Could  you  have  used  a bus  to  make  this  trip? 

G yes  □ no  □ don't  know 

When  was  the  last  time  you  regularly  used  the  bus  to  commute 
from  your  present  home  to  your  present  work  place?  O never 
O not  within  the  last  5 years,  or month  year 


□ 


16 


19 


When  was  the  last  time  you  regularly  used  a carpool  to  commute 
from  your  present  home  to  your  present  work  place?  O never 
O not  within  the  last  5 years,  or month  year 


16.  If  you  do  not  now  regularly  commute  from  home  to  work  by  bus.  why  not?  (Mark  one  or  more  of  the  following) 

□ I now  regularly  commute  by  bus 

□ Bus  not  available 

□ Need  car  during  work  day 

□ Bus  takes  too  long 

□ No  seats  available  on  bus 

□ Bus  unreliable 

□ Too  much  time  spent  waiting  at  but  stops 

□ Too  much  walking  necessary 

□ Bus  too  expensive 

□ No  personal  privacy  on  bus 

□ Other  factor(s)  (specify) 


16.  If  you  do  not  now  regularly  commute  from  home  to  work  by  carpool.  why  not?  (Mark  one  or  more  of  the  following) 

□ I now  regularly  commute  by  carpool 

□ Inability  to  locate  others  willing  to  carpool 

□ Too  much  time  required  to  pick  up  and  discharge  carpool  passengers 

□ Too  much  risk  to  personal  safety 

□ No  personal  privacy  in  carpool 

□ Too  much  auto  insurance  required 

□ Loss  of  flexibility  in  working  hours 

□ Need  car  during  work  day 

□ Other  factor(s)  (specify) 


17.  If  you  could  not  commute  from  home  to  work  by  driving  alone  how  would  you  usually  make  this  trip? 

□ would  be  unable  to  make  this  trip  □ use  bus 

□ join  or  form  a carpool  □ other  (specify)  

18.  What  are  your  regular  working  hours?  □ no  regular  working  hours,  or A.M.  to  P.M. 


19.  When  was  the  last  time  you  changed  your  place  of  residence?  □ not  within  the  last  5 years,  or month  year 

20.  When  was  the  last  time  you  changed  your  physical  work  location?  □ not  within  the  last  5 years,  or month year 

21.  Is  an  auto  available  for  you  to  REGULARLY  drive  alone  from  home  to  work? 

□ no  □ yes.  but  with  considerable  inconvenience  to  others.  □ yes.  and  without  inconvenience  to  others 

22.  How  many  automobiles  are  owned  or  operated  by  members  of  your  household?  □ none,  or auto(s) 

23.  Which  of  the  following  attitudes  best  expresses  your  opinion  about  driving  alone  as  a way  of  commuting  from  home  to  work? 

□ I am  generally  satisfied  with  driving  alone  as  a long  range  solution  to  my  commuting  problems. 

□ I am  generally  satisfied  with  driving  alone  as  a short  term  solution  to  my  commuting  problems  until  improvements  are 
made  in  mass  transit. 

□ I am  generally  dissatisfied  with  driving  alone;  why 

□ Other  (specify) - 

24.  Please  indicate  your  Sea  □ Male  □ Female  Age  □ under  21  □ 21-39  □ 40-65  □ over  65 


62 

□ 

63 


m 

64  65 

□ 


25.  What  is  the  combined  annual  income  of  all  members  of  your  household?  □ SO- 5. 000  □ $5,001-15.000  □ $15,001-30.000  □ Above  $30,000 

26.  Any  comments? 


m 


67  68 


THANK  YOU  - PLEASE  SEAL  AND  MAIL 
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EXHIBIT  A-9 


MINNEAPOLIS  SELF -ADMINISTERED  BUS  SURVEY 


Survey  is  sponsored  by  U.S.  Dept,  of  Transportation 

Please  give  us  a fe-.v  minutes  and  v.-e  will  try  to  give  you  better  Transportation  service. 



Chairman,  Metropolitan  Transit  Commission  qub  CM-R5637 


Commissioner  of  Highways 


1 . I got  on  th 1 s bus  a t 


The  place  I have 
come  from  i s : 


The  place  I a 
going  to  is: 


(Nearest  Street  Cot  ner) 
( Address  or  Street  Intersect- i on) 


Homo?  □ YES 


Work?  □ YES 


(Address,  Building  or  Street  Intersection) 

4.  I began  this  trip  at  about  A.M.;  and  arrived  at  my  destination  about 

5.  What  arc  your  regul  : 


□ NO 

□ NO 
A.M. 


6. 


working  hours? 
How  did  you  get  t n this  bus? 


A.M. 


0 Walked  Blocks  0 Drove  Auto 

0 Drove  Auto/Parked  [_)  Bus  (Route  No.  ) 

7.  After  leaving  >.his  bus,  how  will  you  complete  your  trip? 

0 Walk Blocks  0 Bus  (Route  No. ) 

8.  On  an  average  day  when  you  board  the  bus,  do  you  find  a seat? 

0 .Alt  ays  0 Usually  0 Seldom 

9.  How  did  you  make  this  trip  prior  to  using  this  bus? 


□ A.M. 

□ P.M. 

□ Auto  Passenger 
0 Otne r 


(Specify) 


0 Other 


(Specify) 


0 Never 


0 Drove  ny  cor  ( passengers) 

0 Alternate  driver  in  car  pool 


O An  auto  passenger 
0 Used  another  bus  (Route  No. 
0 Did  not  make  trip 


If  >ou  formerly  traveled  by  car,  what  is  the  major  reason  you  switched  to  the  bus? 

0 Cheaper  0 baler  0 i.ess  Travel  Time  0 More  Dependable  0 Other 
luring  Monday  through  Friday,  how  often  do  you  normally  ride  the  bus? 

O Ri  de  every  day  0 3 or  4 days  0 1 or  2 days  0 Very  rarely 

How  long  have  you  been  using  this  hu<? 

Years  Months 

If  one  transit  improvement  could  be  made,  which  of  the  following  would  you  select? 
(Check  only  one ) 


(Specify) 


O Lower  Fare 
O More  Frequent  Service 
0 Nore  Comfortable  Vehicle 

Other 


O More  Extensive  Service  (Less  walking) 
O Less  Traveling  Time 
f~j  No  Need  to  Transfer 


(Specify) 

How  would  you  rate  your  general  attitude  toward  traveling  to  work  by  bus? 
0Very  positive  0Positive  0Negative  0 Very  negative 


rrrm 


□n 


EE 


n 


□ 


0 

□ 

□ 

r 


□ 

□ 


Now,  just  a few  questions  for  statistical  purposes: 

15.  How  long  have  you  lived  at  your  present  home  address?  Years 

16.  How  many  autos  are  owned  or  operated  by  members  of  your  household? 

0 None  0 One 

17.  Was  a car  available  to  you  for  this  trip? 

8 No  (bus  Is  only  practical  way) 

Yes  (but  with  inconvenience) 

Yes  (but  I prefer  the  bus) 


0 Two 


0 More  than  two  cars 


18.  Are  you:  0Male  0Female 

19.  Are  you  a licensed  driver?  I I Yes 

20.  My  combined  annual  family  income  Is: 


What  is  your  age? 


□ no 


TT 


□ 


0 


□ under  $3,000  □ $5 .000- $7  .999  □$10,000-511  ,999  □$15,000-51  9.999  □$25,000-529,999 

□ $3,000-54,999  □ 58,000-59,959  □$1  2,000-514,999  □$20,000-524  ,999  0 530,000  or  mere 


□ 

m 


THANK  YOU Please  seal  and  mail 


1331 
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EXHIBIT  A-10 


RECOMMENDATIONS  FOR  QUESTIONS  ON  BOARDING 
AND  ALIGHTING  POINTS 
(For  User  Surveys  Only) 

QUESTION  FORMAT 


1.  "Where  did  you  board  this  (vehicle)?" 


Nearest  Street  Intersection 

•k 

2.  "Where  will  you  (did  you)  get  off  this  (vehicle)?" 


Nearest  Street  Intersection 
RESPONSE  CATEGORIES 


Respondent  should  specify  nearest  street  intersection.  Coders 
can  then  translate  street  address  to  codes  representing  bus 
stops  or,  if  a less  fine-grained  analysis  is  required,  zonal 
codes . 

COMMENTS 


Question  format  contains  parentheses  to  indicate  where  site- 
specific  modes  might  be  substituted. 


*l’he  use  of  "will  you"  or  "did  you"  depends  on  whether  the 
survey  is  filled  out  while  the  respondent  is  on  board  the 
vehicle  or  completed  later  and  returned  by  mail. 
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EXHIBIT  A-ll 

RECOMMENDATIONS  FOR  QUESTIONS  ON  TRIP  ORIGIN 
QUESTION  FORMAT 


la.  "Where  did  this  trip  begin?" 


Street  Address,  City,  Zip  Code 


2.  "Is  this  place  (check  one)” 

□ Home 

□ Place  of  employment 

□ School 

□ Retail/commercial  establishment 

□ Social-recreational  facility 

□ Medical  facility 

□ Personal  business  site 

□ Other  (specify)  


RESPONSE  CATEGORIES 

Use  categories  given  under  "Question  Format"  or,  if  the  main 
purpose  of  the  question  is  to  distinguish  work  vs.  nonwork 
trips,  use  the  following  categories: 

□ Home 

□ Place  of  employment 

□ Other 

Respondent  should  specify  street  address.  Coders  can  then 
translate  street  address  to  zonal  codes,  or  addresses  can  be 
geocoded  using  the  Census  Bureau’s  DIME  files  and  ADMATCH 
program . 


A- 30 


EXHIBIT  A-ll  (CONT.) 


Another  option,  for  interview  surveys,  is  to  have  the  inter 
viewer  show  the  respondent  a map  with  numbered  zones  super- 
imposed and  ask  the  respondent  to  identify  the  origin  zone. 

COMMENTS 


The  question  classifying- nature  of  trip  origin,  in  combina- 
tion with  a question  classifying  nature  of  trip  destination 
is  a better  indication  of  trip  purpose  than  a question 
explicitly  asking  trip  purpose,  which  can  be  confusing  to 
persons  making  multiple-purpose  trips. 
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EXHIBIT  A-12 

RECOMMENDATIONS  FOR  QUESTIONS  ON  TRIP  DESTINATION 


QUESTION  FORMAT 

1.  "What  is  (was)  the  final  destination  of  this  trip?" 


Street  Address,  City,  Zip  Code 

2.  "Is  this  place  (check  one)" 

□ Home 

□ Place  of  employment 

□ School 

□ Retail/commercial  establishment 

□ Social-recreational  facility 

□ Medical  facility 

□ Personal  business  site 

□ Other  (specify)  


RESPONSE  CATEGORIES 


Use  categories  given  under  "Question  Format"  or,  if  the  main 
purpose  of  the  question  is  to  distinguish  work  vs.  nonwork 
trips,  use  the  following  categories: 

□ Home 

□ Place  of  employment 

□ Other 

Respondents  should  specify  street  address.  Coders  can  then 
translate  street  addresses  to  zonal  codes,  or  addresses  can 
be  geocoded  using  the  Census  Bureau’s  DIME  files  and  ADMATCH 
program. 

Another  option,  for  interview  surveys,  is  to  have  the  inter- 
viewer show  the  respondent  a map  with  numbered  zones  super- 
imposed, and  ask  the  respondent  to  identify  the  destination 
zone . 

COMMENTS 


The  question  classifying  nature  of  trip  destination,  in  com- 
bination with  a question  classifying  nature  of  trip  origin, 
is  a better  indication  of  trip  purpose  than  a question 
explicitly  asking  trip  purpose,  which  can  be  confusing  to 
persons  making  mult iple -purpose  trips. 
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EXHIBIT  A-13 


RECOMMENDATIONS  FOR  QUESTIONS  ON 
TRIP  START  AND  END  TIMES 

QUESTION  FORMAT 


1.  "What  time  did  you  begin  this  trip?" 

A.M. 

P.M. 

2.  "What  time  did  you  arrive  at  your  destination?" 

A.M. 

P.M. 


RESPONSE  CATEGORIES 


Depending  on  the  survey  objectives,  beginning/ending  times 
can  be  used  as  given  to  compute  total  trip  times,  or  they 
can  be  coded  using  categories  such  as  A.M.  peak,  midday, 
P.M.  peak,  nighttime. 


* With  personal  interviews  onboard  vehicles,  it  is  not  possible 
to  ask  time  of  arrival  at  destination. 
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EXHIBIT  A-14 


RECOMMENDATIONS  FOR  QUESTIONS  ON  ACCESS  MODE 
TO  TRANSIT  VEHICLE 


QUESTION  FORMAT 


1.  "How  did  you  get  from  the  place  where  this  trip  began  to  the 
(place)  where  you  boarded  this  (vehicle)?" 

2.  "How  will  you  (did  you)  get  to  your  destination  after 
leaving  this  (vehicle)?"* 

RESPONSE  CATEGORIES 


U.S.  Census 


Recommended 


Private  auto,  driver 
Private  auto,  passenger 

Bus  or  streetcar 

Subway,  elevated  train,  railroad 
Walked 

Worked  at  home 
Taxi 

Bicycle  or  motorcycle  > 

Other  f 


Park  ' n'  ride 

Carpool 

Kiss  'n'  ride 

Same  (if  relevant, 
add  dial-a-ride) 

Same 

Same 

Omit 

Same 

Other 


COMMENTS 


Question  format  contains  several  parentheses  to  indicate 
where  site- specif ic  modes  and  locations  might  be  substituted 
to  make  the  question  more  relevant.  The  same  principle  ap- 
plies to  the  recommended  response  categories;  the  above  list 
is  suggestive  and  needs  to  be  adjusted  to  site- specif ic  con- 
cerns such  as  measuring  the  number  of  auto  passengers  for 
evaluation  of  a carpool  encouragement  program. 


*The  use  of  "will  you"  or  "did  you"  depends  on  whether  the 
survey  is  filled  out  while  the  passenger  is  on  board  the 
vehicle  or  is  completed  later  and  returned  by  mail. 
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EXHIBIT  A-15 


RECOMMENDATIONS  FOR  QUESTIONS  ON  WHEN 
PRESENT  MODE  WAS  FIRST  USED 


QUESTION  FORMAT 


For  User  Surveys 

"When  did  you  begin,  to  use  (specify  service)  regularly 
for  the  trip  you  are  now  taking?" 

Q not  applicable,  or 

month  year 

For  Non-user  Surveys 

"When  was  the  last  time  you  regularly  used  (specify 
service)  for  the  trip  you  are  now  taking?" 

Q not  within  the  last  5 years,  or 

month  year 

For  Non-user  Surveys  in  Which  Carpoolers  and  Those  Who 

Drove  Alone  are  Given  Separate  Questionnaires 

For  Carpoolers: 

"When  did  you  begin  to  regularly  use  this  carpool  for 
the  trip  you  are  now  taking?" 

□ not  applicable,  or 

month  year 

For  Those  Who  Drove  Alone: 

1.  "When  was  the  last  time  you  regularly  used 

(specify  service)  for  the  trip  you  are  now  taking?" 

Q not  within  the  last  5 years,  or 

month  year 

2.  "When  was  the  last  time  you  regularly  used  a 
carpool  for  the  trip  you  are  now  taking?" 

□ not  within  the  last  5 years,  or 

month  year 


A- 3 5 


1 

EXHIBIT  A-15  (CONT.) 

RESPONSE  CATEGORIES 

Use  categories  given  under  "Question  Format". 

COMMENTS 

It  is  confusing  to  ask  auto  users  who  drive  alone  when  they 
began  to  regularly  drive  alone  in  taking  their  present  trip. 

Therefore,  they  should  be  asked  when  they  last  regularly  used 
other  modes  in  taking  their  present  trip. 
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EXHIBIT  A-16 


RECOMMENDATIONS  FOR  QUESTIONS  ON  FORMER  TRANSPORTATION  MODE 
QUESTION  FORMAT 


"How  did  you  make  this  trip  before  (specify  service) 
was  available?" 


RESPONSE  CATEGORIES 

U.S.  Census 
Private  auto,  driver 


Private  auto,  passenger 


Bus  or  streetcar 

Subway,  elevated  train,  railroad 
Walked 

Worked  at  home 
Taxi 

Bicycle,  motorcycle  l 
Other  * 


Recommended 

Same  (indicate 
total  number  of 
occupants) 

Same  (indicate 
total  number  of 
occupants) 

Same 

Same 

Same 

Omit 

Same 

Other 


COMMENTS 


The  responses  will  have  to  be  tailored  to  include  particular 
local  transportation  alternatives.  For  instance,  it  might 
be  desirable  to  obtain  information  on  former  auto  occupancy 
levels  for  ex-drivers/passengers . 
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Examples  of  attitudinal  questions  appear  throughout  the 
aforementioned  Exhibits  A- 3 through  A-9,  and  also  in 
Exhibits  A- 17  and  A-18.  The  set  of  questions  in  Exhibit  A- 
17  can  be  used  both  to  measure  users’  and  non-users’ 
evaluations  of  the  transit  service  provided  and  the  factors 
that  have  influenced  their  modal  choices.  This  set  of 
questions  can  also  be  used  to  learn  about  the  opinions  of 
the  general  population  regarding  travel  by  alternate  modes. 
Note  that  respondents  are  asked  not  only  for  opinions  about 
different  travel  characteristics  bus  for  a ranking  of  the 
relative  importance  of  these  characteristics.  The  latter 
set  of  questions  is  needed  to  put  the  respondents’  opinions 
about  the  different  travel  characteristics  into  proper 
perspective.  For  example,  if  several  respondents  indicated 
that  "car"  had  a very  high  status  and  "bus"  had  a very  low 
status,  it  might  at  first  appear  that  the  status  of  the 
automobile  might  deter  the  use  of  bus  transit.  However,  the 
responses  would  be  considerably  less  significant  if  these 
same  respondents  indicated  that  the  "status"  travel 
characteristics  was  rather  unimportant  to  them.  The  set  of 
attitudinal  questions  in  Exhibit  A-18  can  be  used  to  obtain 
a profile  of  the  community  in  which  transit  service  is  being 
provided. 

There  are  no  specific  recommendations  for  the  format  of 
attitudinal  questions,  since  the  design  of  such  questions  is 
entirly  dependent  on  the  particular  attitudes  being  measured 
(e.g.,  opinions  of  a very  subjective  item  or  perceptions 
about  items  which  are  independently  measurable)  and  on  the 
overall  survey  context.  However,  the  following  discussion 
presents  some  general  informative  guidelines  regarding  the 
treatment  of  responses  to  attitudinal  questions. 

There  are  three  types  of  response  categories  which  can 
be  used  for  attitudinal  questions:  nominal,  ordinal,  and 
interval  scales.  Nominal  data  consists  of  mutually 
exclusive  categories  with  no  implied  rating  of  the  responses 
(e.g.,  questions  with  "yes", "no"  answers).  Responses  such 
as  "like  very  much,"  "dislike,"  "dislike  very  much" 
represent  ordinal  level  data,  with  an  implied  rank  ordering. 
Interval  data  involves  the  use  of  numerical  scales  (e.g., 
asking  people  to  indicate  their  opinions  on  a scale  of  1 to 
5) . Since  interval  scales  require  prior  validation  and 
careful  application,  it  is  recommended  that  attitudinal 
questions  be  limited  to  nominal  or  ordinal  response 
categories.  Moreover,  it  is  recommended  that  the  survey 
data  be  represented  in  the  form  of  frequency  distributions, 
rather  than  statistics  such  as  means  which  have  an  implied 
ranking. 
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EXHIBIT  A- 17 

SET  OF  ATTITUDINAL  QUESTIONS  ON  TRAVEL  BY  TRANSIT  AND  AUTO 


YOUR  OPINION  OF  TRAVEL  BY  DIFFERENT  MODES 

4.  On  the  scales  below,  please  indicate  your  general  opinion  of  car  and  bus  travel 
for  local  travel . Base  your  opinion  on  what  you  have  experienced  or  have 

heard  about  local  travel  by  each  mode  from  the  user's  viewpoint.  Even  though  you 
may  not  use  the  bus,  you  probably  have  some  perceptions  of  what  this  form  of 
travel  is  like;  you  don't  need' to  have  tried  something  in  order  to  be  able  to 
express  some  general  opinions. 

To  indicate  your  opinion,  look  at  the  descriptive  scales  below,  each  of  which 
allows  for  a range  of  opinions  on  a particular  characteristic,  such  as  COMFORT. 
Then,  mark  what  you  consider  to  be  the  single  most  appropriate  description  on  each 
scale  by  circling  the  relevant  number.  For  instance,  on  the  COMFORT  scale,  if  you 
thought  cars  were  a very  comfortable  form  of  travel  for  local  travel,  you 
would  circle  "1"  on  the  scale  on  the  line  for  cars;  however,  if  you  thought  they 
were  a slightly  uncomfortable  form  of  travel,  you  would  circle  "4",  end  so  forth. 

Neither  or 


COST  OF  TRAVEL 

Inexpensive 

Car 

1 

2 

3 

4 

5 

Expensive 

Bus 

1 

2 

3 

4 

5 

ENJOYABLE NESS 

Enjoyable 

Car 

1 

2 

3 

4 

5 

Unenjoyable 

Form  of  Travel 

Bus 

1 

2 

3 

4 

5 

Form  of  Travel 

SPEED  ON  NON- 

Fast 

Car 

1 

2 

3 

4 

5 

Slow 

COMMUTE  TRIPS 

Bus 

1 

2 

3 

4 

5 

CONVENIENCE 

Convenient 

Car 

1 

2 

3 

4 

5 

Inconvenient 

Form  of  Travel 

Bus 

1 

2 

3 

4 

5 

Form  of  Travel 

STATUS 

High  Status 

Car 

1 

2 

3 

4 

5 

Low  Status 

Form  of  Travel 

Bus 

1 

2 

3 

4 

5 

Form  of  Travel 

SPEED  ON 

Fast 

Car 

1 

2 

3 

4 

5 

Slow 

COMMUTE  TRIPS 

Bus 

1 

2 

3 

4 

5 

COMFORT  (Seats, 

Comfortable 

Car 

1 

2 

3 

4 

5 

Uncomfortable 

Noise,  Ride,  etc.) 

Bus 

1 

2 

3 

4 

5 

MODERNITY 

Modern  Form 

Car 

1 

2 

3 

4 

5 

Old-Fashioned 

of  Travel 

Bus 

1 

2 

3 

4 

5 

Form  of  Travel 

SAFETY 

Safe  Form 

Car 

1 

2 

3 

4 

5 

Dangerous 

of  Travel 

Bus 

1 

2 

3 

4 

5 

Form  of  Travel 

SIMPLICITY 

Simple  to  Use 

Car 

1 

2 

3 

4 

5 

Complicated 

Bus 

1 

2 

3 

4 

5 

to  Use 

PUNCTUALITY 

On-Time  Arrivals 

Car 

1 

2 

3 

4 

5 

Late  Arrivals 

Bus 

1 

2 

3 

4 

5 
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EXHIBIT  A- 17  (CONT.) 


YOUR  RANKING  OF  THE  RELATIVE  IMPORTANCE  OF  THE  DIFFERENT  TRAVEL  CHARACTERISTICS 

Listed  below  are  the  set  of  travel  characteristics  which  appeared  in  the 
previous  question.  In  the  blanks  next  to  each  characteristic,  please 
indicate  how  important  that  characteristic  is  in  your  decision  to  use  car 
or  bus  for  local  travel,  by  placing  one  of  the  following  "importance" 
numbers  in  the  blank: 

3 - Very  important 

2 - Moderately  important 

1 - Unimportant,  or  don't  consider  it 


Cost  of  Travel 
Enjoyableness 

Speed  on  non-commute  trips 

Convenience 

Status 

Speed  on  commute  trips 

Comfort,  (seats,  noise,  ride,  etc.) 

Modernity 

Safety 

Simplicity 

Punctuality 
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EXHIBIT  A-18 

SET  OF  QUESTIONS  ON  GENERAL  ATTITUDES  OF  THE  POPULATION 


YOUR  OPINIONS  ABOUT  LOCAL  GOVERNMENT 


Everyone  has  different  ideas  about  the  kinds  of  things  local  government  should  be 
most  concerned  about.  Below  is  a list  of  different  things  the  government  might 
do.  Please  indicate  your-  feeling  about  how  much  the  government  should  do  of 
each  activity. 

About 


Government  Activities 

Much 

more 

Slightly 

more 

the 

same 

Slightly 

less 

Much 

less 

i) 

Reduce  crime.  .....  ... 

m 

s 

1 

L 

ii) 

Reduce  environmental  pollution.  . . . . 

m 

s 

1 

L 

iii) 

Provide  low-cost  medical  care  for  all  . 

. M 

m 

s 

1 

L 

iv) 

Control  population  growth  ....... 

m 

s 

1 

L 

v) 

Provide  more  housing  for  low  to  medium 
income  families  ............ 

m 

s 

1 

L 

vi) 

Insure  equal  opportunity  for  women.  . . 

. M 

m 

s 

1 

L 

vii) 

Provide  consumer  protection  ...... 

m 

s 

1 

L 

viii) 

Add  to  and  improve  the  freeway  system  . 

. M 

m 

s 

1 

L 

ix) 

Increase  direct  aid  to  the  poor  . . . . 

m 

s 

1 

L 

x) 

Improve  bus  service  and  other  forms  of 
public  transportation  ......... 

. M 

m 

s 

1 

L 

xi) 

Have  more  parks  and  outdoor  recreation 
areas  ........ 

m 

s 

1 

L 

xii) 

Improve  the  public  schools . 

m 

s 

1 

L 

xiii) 

Reduce  taxes.  ......  .. 

m 

s 

1 

L 

(a) 

Which  one  of  the  activities  do  you  feel 

is  the 

most  important 

for  the  government 

(b) 

to  do?  Just  give  the  letter. 

And  which  do  you  feel  is  the  next  most 

Most 

important? 

Next 

important 
most  important 
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EXHIBIT  A-18  (CONT.  ) 


YOUR  OPINIONS  ON  TRANSPORTATION  AND  PERSONAL  TRAVEL 

Below  are  listed  a number  of  statements  relating  to  transportation  facilities  and 
personal  travel;  you  will  probably  agree  with  some  of  them  and  disagree  with  others. 
Please  answer  by  circling  the  letter  which  best  represents  your  feeling  about  each 


of  the  statements,  according  to  the  following  codes: 

A a o d D 

means  means  means  neither  means  means 

Strongly  Agree  Agree  Somewhat  Agree  nor  Disagree  Disagree  Somewhat  Strongly  Disagree 

I much  prefer  driving  a car  to  being  a passenger  in  one A a o d D 

It's  time  measures  were  taken  to  discourage  auto  usage  in  downtown  A a o d D 

I really  can't  see  much  of  a future  for  public  transportation A a o d D 

I could  manage  without  a car  for  a few  months  if  I had  to A a o d D 

People  would  use  public  transportation  a lot  more  if  fares  were  lower A a o d D 

I'd  much  rather  people  saw  me  arriving  at  work  by  car  than  getting  off  a bus A a o d D 

I've  never  really  bothered  to  find  out  details  of  what  public  transportation 

services  are  available  around  here A a o d D 

A lot  of  my  friends  and  acquaintances  judge  people  by  the  type  of  car  they  drive. . .A  a o d D 

It's  important  that  my  home  be  close  to  good  public  transportation  services A a o d D 


Government  investments  in  mass  transit  are  a good  way  to  help  reduce  air  pollution. A a o d D 


I've  got  bad  memories  of  public  transportation A a o d D 

Everyone  has  a right  to  drive  his  car  just  as  much  as  he  wants A a o d D 

Public  transportation  is  no  use  at  all  for  journeys  outside  commute  hours A a o d D 

I enjoy  driving  very  much A a o d D 

It  would  hardly  seem  proper  for  someone  in  a top  job  to  commute  by  bus A a o d D 

I hate  to  be  tied  to  fixed  schedules  for  traveling A a o d D 

I might  use  public  transportation  more  often  if  it  were  simpler  to  obtain 

information  about  routes  and  schedules A a o d D 

Traveling  by  public  transportation  is  so  much  more  relaxing  than  driving A a o d D 

I often  worry  about  being  involved  in  a bad  car  accident A a o d D 

I'd  never  travel  regularly  by  any  form  of  public  transportation,  no  matter  how 

much  they  improved  the  service A a o d D 

The  idea  of  carpooling  doesn't  appeal  to  me A a o d D 

There  should  be  a greater  emphasis  on  developing  improved  public  transportation 
systems  and  less  on  building  freeways A a o d D 

I'm  always  glad  of  an  excuse  to  take  my  car  out  for  a drive A a o d D 
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In  the  past,  methods  for  categorizing  responses  to 
attitudinal  questions  have  been  varied  and  inconsistently 
applied.  Accordingly,  these  guidelines  cannot  offer  a 
preferred  means  of  categorization  for  each  measure  which 
might  arise.  As  more  experience  is  gained  in  this  area 
through  independent  research*  and  future  demonstrations,  TSC 
will  attempt  to  develop  a consistent  approach  toward  the 
application  of  attitudinal  measures  and  the  categorization 
of  responses. 


(3)  Social  and  Demographic  Measures 

The  inclusion  of  certain  social/demographic  questions 
in  surveys  serves  the  dual  purpose  of  1)  providing  data  on 
respondent  characteristics  which  might  show  a correlation 
(perhaps  even  a causal  relationship)  with  measured 
behavioral  attributes,  and  2)  providing  data  about 
respondents  which  can  be  used  in  conjunction  with  Census 
data  to  check  survey  accuracy,  determine  non-response  bias, 
and  extrapolate  survey  findings  to  other  areas. 

The  amount  and  nature  of  social/demographic  information 
collected  depends  on  a number  of  factors,  in  particular,  the 
desired  length  of  the  survey  and  the  extent  to  which  the 
data  will  be  correlated  with  behavioral  data  and  used  for 
extrapolation  purposes.  It  is  recommended  that  the 
following  items  be  included  in  every  survey:  respondent's 
sex,  age,  household  income,  the  number  of  autos  in  the 
respondent's  household,  and  availability  of  an  auto  for  the 
particular  trip(s)  made  on  transit  (user  surveys  only). 
Depending  on  the  survey  objectives,  scope,  and 
administration  format,  the  following  are  some  of  the 
additional  items  which  might  be  included:  whether  the 
respondent  has  a driver's  license,  the  general  (regular) 
availability  of  an  auto  for  a particular  trip  type  (e.g., 
work) , educational  level  completed,  occupation,  and  length 
of  residence  and  employment  at  present  location. 

Examples  of  questions  on  social/demographic  variables 
appear  throughout  Exhibits  A- 3 through  A-9.  Exhibits  A- 19 
through  A- 2 8 present  the  recommended  question  format  and 


*For  example,  the  forthcoming  UMTA/TSC- sponsored  research 
on  the  predictive  validity  of  attitudinal  surveys  should 
be  helpful  in  this  regard. 
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EXHIBIT  A-19 

RECOMMENDATIONS  FOR  QUESTIONS  ON  RESPONDENTS’  SEX 
QUESTION  FORMAT 

A.  For  Self-Administered  Surveys 


"Are  you  

□ Male 

□ Female 

or 

’’Please  indicate  your  sex" 

□ Male  □ Female 

B.  For  Interview  Surveys 

Respondent’s  sex  is  noted  by  the  interviewer. 
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EXHIBIT  A-20 


RECOMMENDATIONS  FOR  QUESTIONS  ON  RESPONDENT’S  AGE 
QUESTION  FORMAT 


"To  what  age  group  do_  you  belong?" 


Categories  (see  below) 

: / 


RESPONSE  CATEGORIES 


U.S.  Census 

Under  5 

5-9 

10-14 

15-19 

20-24 

25-29 

30-34 

35-39 

40-44 

45-49 

50-54 

55-59 

60-64 

65-69 

70-74 

75-79 

80-84 

85  and  over 


Recommended 
Under  20 

20-44 

45-64 

65  and  over 


COMMENTS 


The  recommended  response  categories  represent  the  minimum 
stratification  of  data  to  be  collected  about  age.  Age 
responses  can  be  further  stratified  according  to  the  U.S. 
Census  categories,  depending  on  the  survey  objectives  and 
the  expected  age  distribution  of  the  respondent  population. 

It  is  important  to  use  the  phrase  "age  group"  in  all  ques- 
tions about  age  to  minimize  the  respondent's  resistance  to 
this  question. 
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EXHIBIT  A-21 

RECOMMENDATIONS  FOR  QUESTIONS  ON  RESPONDENT'S  INCOME 
QUESTION  FORMAT 


"What  is  the  combined  annual  income  of  all  members  of 
your  household?" 


Categories 


(see  below) 


RESPONSE  CATEGORIES 


U.S.  Census 

Less  than  $1,000 
$1,000  - 1,999 
$2,000  - 2,999 
$3,000  - 3,999 
$4,000  - 4,999 
$5,000  - 5,999 
$6,000  - 6,999 
$7,000  - 7,999 
$8,000  - 9,999 
$10,000  - 14,999 
$15,000  - 24,999 
$25,000  and  over 


Recommended 
Less  than  $5,000 

$5,000  - 9,999 

$10,000  - 14,999 
$15,000  - 24,999 
$25,000  and  over 


COMMENTS 


The  recommended  response  categories  represent  the  minimum 
stratification  of  income  data.  Responses  can  be  further 
stratified  according  to  the  U.S.  Census  categories,  depend- 
ing on  the  survey  objectives  and  the  expected  income  dis- 
tribution of  the  respondent  population. 

For  interview  surveys,  asking  a respondent  to  point  to  one 
of  the  above  categories  on  a card  facilitates  handling  of 
this  often  sensitive  question. 

It  is  important  to  use  the  word  "annual"  or  "yearly"  in  order 
to  obtain  responses  on  a consistent  basis.  Moreover,  if 
deemed  appropriate,  the  question  can  be  phrased  to  refer  to 
the  most  recently  ended  calendar  year. 
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EXHIBIT  A-22 


RECOMMENDATIONS  FOR  QUESTIONS  ON  AUTO  AVAILABILITY 

(For  User  Surveys  Only) 

QUESTION  FORMAT 


"Was  a car  available  to  you  for  this  trip?" 


RESPONSE  CATEGORIES 

"Was  a car  available....?" 

O Yes,  and  without  inconvenience  to  others. 
D Yes,  but  with  inconvenience  to  others. 

D No. 


COMMENTS 


Information  on  the  availability  of  a car  for  a specific  trip 
or  time  period  is  the  most  direct  way  of  determining  auto 
availability  and  its  possible  influence  on  mode  used. 
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EXHIBIT  A-2  3 


RECOMMENDATIONS  FOR  QUESTIONS  ON  AUTO  OWNERSHIP 
QUESTION  FORMAT 


"How  many  cars  are  owned  or  operated  by  members  of 
your  household?" 

RESPONSE  CATEGORIES 


U.S.  Census  Recommended 

0 cars  □ None,  or  auto(s) 

1 car 

2 cars 

3 or  more  cars 


EXHIBIT  A-24 


RECOMMENDATIONS  FOR  QUESTIONS  ON  WHETHER  RESPONDENT  HAS 

DRIVER’S  LICENSE 


QUESTION  FORMAT 


"Are  you  a licensed  driver?" 

RESPONSE  CATEGORIES 
"Are  you  . . . . " 

□ Yes  □ No 
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EXHIBIT  A-2 5 

RECOMMENDATIONS  FOR  QUESTIONS  ON  RESPONDENT’S  OCCUPATION 

QUESTION  FORMAT 

1.  "Are  you  " 

Q Employed 
D Student 
Q House  Spouse 
D Retired 
D Other 

2.  "If  you  are  employed,  describe  briefly  the  kind  of 

work  you  do." 


CODING  CATEGORIES  FOR  QUESTION  2 
U.S.  Census 

Professional,  technical  and 
kindred  workers 
Managers  and  administrators, 
except  farm 
Salesworkers 

Clerical  and  kindred  workers 
Craftsmen  and  kindred  workers 
Operatives,  except  transport 
Transport  equipment  operatives 
Laborers,  except  farm 
Farmers  and  farm  managers 
Farm  laborers  and  farm  foremen 
Service  workers,  except  private 
household 

Private  household  workers 

COMMENTS 


Recommended 

The  survey  form 
should  contain  a 
blank  space  for  an 
open-ended  description 
which  can  later  be 
coded  using  the 
U.S.  Census  occupa- 
tional categories. 


Question  2 should  be  included  in  the  survey  only  when 
a very  specific  reason  for  using  employment  data.  In 
perform  the  coding  for  question  2,  it  is  necessary  to 
description  of  the  type  of  work  actually  done  as  well 
title . 


there  is 
order  to 
obtain  a 
as  j ob 
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EXHIBIT  A-26 


RECOMMENDATIONS  FOR  QUESTIONS  ON  RESPONDENT’S 
EDUCATIONAL  LEVEL 


QUESTION  FORMAT 


For  All  Surveys 

"What  is  the  last  grade  (or  year)  of  regular  school 
you  (he/she)  attended?" 


Categories 


(see  below) 


(asked  for  each  household  member  in  dwelling  unit  survey) 


RESPONSE  CATEGORIES 
U.S.  Census 

None 

1-4 

5-7 

8 

High  School  : 1-3 

4 

College:  1-3 

4 years  or  more 


Recommended 

No  Formal  Schooling 

Grade  School 

Some  High  School 
High  School  Degree 
Some  College 

College  degree  or  higher 
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EXHIBIT  A-27 

RECOMMENDATIONS  FOR  QUESTIONS  ON  LENGTH  OF  RESIDENCE 


QUESTION  FORMAT 


"When  did  you  (your  household)  move  to  your  present 
residence?" 


RESPONSE  CATEGORIES 

U.S.  Census* *  Recommended 

1969-70  fj  Not  within  the  last  5 years, 

1968 

1967  month 

1965-66 

1960-64 

1950-59 

1949  or  earlier 
COMMENTS 


The  recommended  response  categories  represent  the  minimum 
stratification  of  data  to  be  collected  about  length  of 
residence.  Responses  can  be  further  stratified  (for  greater 
than  5 years),  depending  on  the  survey  objectives  and  the 
expected  residency  level  distribution  of  the  respondent 
population. 


*The  U.S.  Census  categories  reflect  the  year  of  the  Census, 
1970. 


or 

year 
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EXHIBIT  A-28 


RECOMMENDATIONS  FOR  QUESTIONS  ON 
TRIP  PRODUCTION  RATES 


QUESTION  FORMAT 


"Please  indicate  what  trips  you  have  recently  taken." 
RESPONSE  CATEGORIES 


Trip  Purpose 

Work 

School 

Shopping 

Social-Recrea  tional 
Medical -Dental 
Personal  Business 
Other 


(Count  the  number  of  one-way  trips  you  took  under  each  pur- 
pose. If  you  went  shopping  and  then  returned  home, 
that  is  two  (2)  shopping  trips.  If  you  took  a trip  with 
multiple  purposes  count  it  under  the  principal  purpose  of 
the  trip  and  mark  a check  (✓)  for  any  other  purposes  served.) 
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response  categories  for  most  of  the  social/demographic 
measures  listed  above.  The  recommended  categories 
deliberately  parallel  U.S.  Census  categories  where 
possible.  It  is  considered  important  to  collect  and  code 
this  type  of  data  in  categories  which  are  eguivalent  to,  or 
collapsible  into.  Census  categories,  so  as  to  facilitate 
comparisons  with  the  same  type  of  Census  data  for  the  survey 
area  (for  accuracy  check  purposes) *,  or  to  permit  the  use  of 
other  types  of  Census  data  to  amplify  survey  findings  (with 
the  collected  data  serving  as  a bridge  between  the  survey 
population  and  the  Census  population)  . Special  purpose 
surveys  may  require  a greater  amount  of  detail  about  a 
particular  social/demographic  measure,  but  the 
stratification  should  be  compatible  with  commonly  used 
Census  breakdowns.** 


6.  NON-RESPONSE  BIAS 

Use  of  the  guidelines  presented  in  this  Appendix  to 
design  and  execute  a survey  does  not  insure  that  the 
responses  obtained  will  accurately  reflect  the 
characteristics,  travel  behavior,  and/or  attitudes  towards 
the  demonstration  project  service  of  the  entire  sample 
selected  even  though  the  sample  itself  is  unbiased  and 
totally  representative  of  the  population  from  which  the 
sample  was  selected.  It  is  possible  that  the 
characteristics,  behavior,  and  attitudes  of  the  part  of  the 
sample  that  did  not  respond  to  the  survey  are  different  from 
those  of  the  part  that  did  respond,  hence  producing  non- 
response bias. 

Pretesting  of  the  survey  instrument  may  or  may  not 
reveal  this  problem  when  it  exists.  Even  if  pretesting  does 
reveal  the  problem,  there  may  be  no  effective  means  of 
eliminating  it.  This  is  especially  true  if  there  is  a 


*Census  tract  or  block  data  on  family  income  will  be  a good 
check  on  reporting  accuracy. 

**See  U.S.  Census,  Volume  I:  Characteristics  of  the 

Population,  Part  II,  Appendix  B for  a detailed  discussion 
on  format  of  questions.  See  also  "1970  Census  User  Guide," 
U.S.  Department  of  Commerce,  Bureau  of  the  Census, 
Washington,  D.C.,  June,  1969. 
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systematic  refusal  to  participate  in  a survey  by  certain 
segments  or  personality  types  in  the  population.  It  is 
recommended  here  that  an  attempt  be  made  in  every  survey  to 
determine  whether  or  not  non-response  bias  exists  and  how  it 
might  affect  the  validity  of  results. 

There  are  no  specific  guidelines  for  ascertaining  the 
existence  of  non-response  bias.  In  general,  non-respondents 
can  be  reached  with  a very  short  survey  containing  but  a few 
key  questions  that  is  administered  with  considerable 
personal  contact.  Where  non-respondents  cannot  be 
identified,  the  special  survey  would  be  given  with  the 
regular  survey  to  a part  of  the  sample.  Many  of  those  who 
do  not  respond  to  the  regular  survey  will  respond  to  the 
special  survey.  Where  non-respondents  can  be  identified 
after  the  regular  survey  has  been  completed,  only  they  would 
be  given  the  special  survey.  The  responses  of  respondents 
and  non-respondents  to  the  few  key  questions  can  then  be 
compared  to  determine  whether  the  responses  of  respondents 
and  non-respondents  are  significantly  different,  and 
therefore,  whether  non-response  bias  exists. 

The  contractor  should  attempt  to  devise  a specific 
methodology  for  determining  whether  non-response  bias  exists 
in  the  survey  responses  obtained  from  the  surveys  being 
conducted. 


7.  INTERVIEWS  WITH  TRANSIT  COMPANY  PERSONNEL 

There  are  situations  where  it  may  be  useful  to  conduct 
interviews  with  transit  company  personnel,  e.g.,  drivers, 
mechanics,  management.  In  some  cases,  such  interviews  could 
be  used  to  develop  ideas  for  questions  and  sets  of  responses 
for  surveys  of  users  and  non-users.  This  is  useful  in 
situations  where  changes  are  being  hypothesized,  and  transit 
company  personnel  could  give  their  opinions  and  insight  on 
measuring  these  potential  changes. 

In  other  cases,  such  interviews  could  be  used  to  check 
the  validity  of  collected  data  and  survey  responses.  In 
some  very  specific  cases,  such  interviews  could  provide 
first-hand  data  on  certain  project  characteristics.  For 
example,  drivers  and  mechanics  could  provide  information  on 
the  operating  and  maintenance  characteristics  of  an 
innovative  vehicle,  e.g.,  double  deck  bus.  Management  could 
provide  insight  into  the  labor  problems  associated  with 
operating  a handicapped  and  elderly  project.  Dispatchers 
could  provide  insight  into  the  operating  characteristics  of 
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a demand-responsive  system.  The  situations  discussed  are 
not  meant  to  be  all-inclusive.  No  specific  guidelines  have 
been  put  forth.  It  is  up  to  the  contractor  to  decide 
whether  interviews  with  transit  company  personnel  would 
provide  information  needed  to  perform  the  particular 
evaluation,  and  to  design  the  appropriate  survey  technique. 


APPENDIX  B 


STATISTICAL  METHODOLOGY 


This  Appendix  presents  guidelines  relevant  to 
determining  appropriate  sample  sizes  for  data  collection  as 
well  as  the  subsequent  analyses  based  on  sample  data. 

The  determination  .of  appropriate  sample  sizes  and  data 
analysis  requirements  is  a crucial  aspect  of  planning  for 
data  collection,  since  in  general  this  phase  involves 
scoping  the  level  of  activity  related  to  collection  of 
project-specific  measures.  Just  as  failure  to  plan  the 
basic  evaluation  approach  will  mean  not  having  the  proper 
framework  in  which  to  observe  and  evaluate  the 
demonstration,  failure  to  plan  or  improper  planning  of 
sample  size  requirements  and  data  analysis  procedures  will 
threaten  the  ultimate  statistical  validity  and  usefulness  of 
project  results.  An  insufficient  quantity  of  data  (whether 
due  to  no  planning,  i.e.,  haphazard  data  collection,  or  to 
an  underestimate  of  needs)  will  be  manifested  in  the  loss  of 
potentially  valuable  analyses  and/or  a loss  in  accuracy  and 
validity  of  the  analyses  based  on  the  data.  On  the  other 
hand,  excessive  quantities  of  data  will  mean  the  unnecessary 
expenditure  of  funds  and  possibly  the  sacrifice  of  other 
data  items  which  could  be  useful  but  which  are  beyond  a 
constrained  budget.  The  ideal  is  to  obtain  an  appropriate 
balance  between  analysis  requirements  and  resource 
availability.  It  should  be  remembered  that  small  samples, 
if  they  are  well  planned,  can  yield  useful  and  interpretable 
data. 


1.  DEFINITIONS 

To  assure  a complete  understanding  of  the  concepts 
presented  in  this  Appendix,  as  well  as  those  identified  in 
the  references  thereto,  the  following  terms  are  identified: 

(1)  Observational  entity  or  element  - An  individual 
item  in  a set  of  items  or  responses,  each  of  which 
is  identifiable  by  one  or  more  measures.  Examples 
of  observational  entities  are  automobiles, 
vehicles,  persons,  time  periods. 

(2)  Population  or  universe  - A finite  or  perhaps  very 
large  collection  of  observational  entities.  A 
population  is  usually  a group  about  which 
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inferences  are  desired.  Examples  of  populations 
would  be  all  those  vehicles  on  a corridor  leading 
to  the  central  business  district  during  AM  peak 
periods,  all  those  persons  within  15  minutes 
access  time  of  the  transit  system,  or  all  users  of 
a service. 

(3)  Sample  - A finite  subset  of  observational  entities 
drawn  from  a population.  Samples  can  be  drawn  by 
appropriate  procedures  which  will  permit 
inferences  to  the  population  from  which  the  sample 
was  drawn  or  they  may  be  obtained  by  non- 
controlled  devices.  Examples  of  samples  would  be 
some  of  the  vehicles  passing  a given  screen-line 
during  a specific  time  period,  or  a subset  of 
those  individuals  within  a service  area. 

(4)  Observation  - One  or  more  measures  which  describe 
the  observational  entities  included  in  the  sample 
either  directly  or  derived  from  measurements,  such 
as  travel  times  or  passenger  counts. 

(5)  Population  parameter  - A specific  descriptive 
characteristic  of  a population  assumed  to  be 
constant  at  any  moment  or  period  in  time. 

(6)  Sample  statistic  - A summary  value  obtained  from  a 
sample  observation,  usually  descriptive  of  the 
sample  but  desired  for  purposes  of  making 
inferences  about  the  population  or  changes  in  the 
population  parameter. 


2.  DATA  ANALYSIS  DETERMINATION 

It  should  be  evident  that  the  intent  of  using  samples 
is  to  make  inferences  about  changes  in  transit  system 
characteristics  or  in  the  attitudinal/behavioral 
characteristics  of  the  community  being  served. 

Before  estimating  sample  size  requirements,  it  is 
necessary  to  determine  the  appropriate  types  of  analyses  to 
be  performed  (i.e..  What  will  be  done  with  the  data  once  it 
has  been  collected?) . Types  of  statistical  analyses  which 
can  be  performed  are  numerous.  As  a general  guideline,  it 
is  essential  that  the  evaluations  for  SMD  projects  be 
confined  to  fairly  fundamental  types  of  analyses — i.e., 
involving  the  calculation  of  means,  standard  deviations  or 
variances,  proportions,  ratios,  and  ranges.  Suggested 
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statistical  techniques  for  performing  these  analyses  are 
discussed  later  in  this  Appendix. 

It  should  be  noted  that  more  sophisticated  statistical 
methods,  such  as  multiple  regression,  factor  analysis,  and 
discriminant  analysis,  while  highly  useful  in  a theoretical 
sense,  may  not  be  applicable  in  the  current  generation  of 
SMD  projects.  As  more  experience  is  gained  with  the  data 
collected  during  these  demonstrations,  it  may  be  possible  to 
institute  some  of  the  referenced  multivariate  techniques. 

The  use  of  a simple  analytical  framework  will  have 
three  main  advantages:  (1)  the  results  will  be  expressed  in 

numerical  terms  that  have  a direct  relation  to  specific 
project  objectives;  (2)  the  evaluation  results  will  be 
meaningful  to  a wide  audience;  and  (3)  the  results  of  a 
particular  demonstration  can  be  more  easily  compared  with 
those  of  other  projects. 

The  types  of  statistical  analyses  which  can  be 
performed  and  the  appropriate  equations  and  tables  to  be 
used  in  performing  these  analyses  and  determining  sample 
sizes  are  presented  in  an  organized,  thoroughgoing  manner  in 
M.G.  Natrella,  Experimental  Statistics,  National  Eureau  of 
Standards  Handbook  91,  August  1,  1963.*  Included  in  this 
handbook  are  procedures  for  estimating  average  performance 
from  a sample,  estimating  variability  of  performance  for  a 
sample,  comparing  two  or  more  samples  with  respect  to 
average  performance  or  variability  of  performance, 
characterizing  the  functional  relationship  between  two 
variables,  and  comparing  samples  with  respect  to  discrete 
classifications  such  as  income,  mode  of  travel  to  work,  etc. 
Two  other  excellent  references  are  given  at  the  end  of  this 
Appendix.  Since  most  of  the  specific  equations  to  be 
employed  in  dealing  with  these  situations  are  clearly 
presented  in  Natrella  and  other  commonly  used  statistics 
reference  books,  the  remainder  of  this  section  will  be 
devoted  primarily  to  a discussion  of  some  of  the  statistical 
considerations  by  the  contractor. 


*The  contractor  is  encouraged  to  obtain  a copy  of  this  book, 
since  it  is  referenced  throughout  this  section  of  the  guide- 
lines as  a source  for  tables,  equations  and  other  materials. 
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Of  the  numerous  cases  presented  in  Natrella,  the 
following  basic  set  of  underlying  questions  is  considered 
applicable  for  SMD  evaluations: 

If  estimates  of  population  parameters  only  are 

required: 

(1)  What  is  an  estimate  for  the  average  value  (mean) 
of  the  measure  (let  X represent  the  measure)? 

(2)  What  is  an  estimate  for  the  variability  (variance 
or  standard  deviation)  of  the  measure? 

(3)  What  is  an  estimate  of  the  proportion  of  units 
that  have  a given  characteristic? 

If  comparisons  between  two  groups  (e.  g. r before  vs. 

after;  test  vs . control)  are  involved : 

(1)  What  is  the  difference  between  the  average  value 
of  the  measure,  X,  for  group  A and  the  average 
value  of  the  measure,  X,  for  group  B? 

(2)  Same  question  as  (1)  except  applied  to  the 
variability  of  the  measure  in  groups  A and  B. 

(3)  Same  question  as  (1)  except  applied  to  proportions 
of  some  discrete  measure  in  groups  A and  B. 

The  same  types  of  questions  can  be  phrased  when  there 
are  more  than  two  groups  (time  periods)  involved  in  the 
comparisons.  Here,  however,  the  methods  for  analysis  become 
more  complex,  and  greater  care  must  be  exercised  in 
selecting  and  applying  statistical  techniques. 

In  connection  with  addressing  the  question  "What  is  the 
value...?  or  "What  is  the  difference...?",  it  is  recommended 
that  results  be  given  in  terms  of  confidence  intervals 
rather  than  tests  of  significance.  By  presenting  a 
confidence  interval  (an  interval  which  contains  the  true 
parameter,  or  difference  between  two  parameters,  with  a 
known  probability) , the  decision-maker  can  interpret  the 
magnitude  of  this  interval  whether  it  be  for  an  estimate  of 
a population  parameter  or  for  the  difference  between  two 
parameters.  On  the  other  hand,  if  a test  of  significance  is 
used,  the  interpretation  of  non- significance  and 
significance  becomes  somewhat  more  difficult  in  terms  of 
relating  these  inferences  back  to  the  project  objectives. 

In  some  instances  where  sample  sizes  are  fairly  large, 
differences  that  can  be  significant  from  a statistical 
viewpoint,  may  have  little  practical  significance  attached 
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to  them.  Statements  on  statistical  significance  may  be  made 
but  the  practical  implications  must  be  considered. 

It  will  generally  be  adequate  for  the  contractor  to 
report  two-sided  confidence  intervals  for  a stated 
confidence  level. 


3.  SAMPLE  SIZE  DETERMINATION 

As  long  as  appropriate  sampling  methods  are  applied, 
the  accuracy  of  a statistic  computed  from  a sample  will  be 
greater  with  a larger  sample  size.  However,  this 
relationship  can  be  one  of  diminishing  returns  for  very 
large  sample  sizes.  Moreover,  there  is  a cost,  in  time  and 
money,  which  serves  as  a constraint  on  sample  sizes  in  each 
demonstration  project.  The  key  aspect  of  sample  size 
determination  is  finding  the  proper  balance  between  desired 
accuracy  and  cost:  on  the  one  hand,  the  sample  should  not  be 
so  small  that  the  results  lack  the  required  accuracy; 
conversely,  the  sample  should  not  be  wastefully  large. 

In  Chapter  III,  Section  D,  variable  stratification  (the 
categorization  of  collected  data  by  such  factors  as  time  of 
day)  was  discussed.  It  was  mentioned  that  the  data 
collection  activities  should  be  planned  with  the  finest 
level  of  stratification  consistent  with  constraints  of  time, 
cost,  and  acceptable  accuracy  and  confidence.  It  is 
important  that  this  determination  of  desired  level  of 
stratification  be  made  as  early  as  possible,  since,  from  the 
statistical  point  of  view,  the  sampling  plans  must  include 
sufficient  data  in  each  category  of  interest  for  which 
cross-tabulations  are  to  be  performed.  The  formulas  for 
determining  sample  size  must  be  applied  with  respect  to  each 
category,  so  that  the  appropriate  quantity  of  data  is 
collected  for  each  one.  Clearly,  an  attempt  at  further 
stratification  after  the  data  has  been  collected  would 
reduce  the  accuracy  and/or  confidence  associated  with  these 
new  sub-stratifications. 

The  appropriate  sample  size  formula  depends  on  the  type 
of  statistical  analysis  to  be  performed.  Sample  size 
formulas  applicable  for  calculating  means,  variances, 
proportions,  etc.,  are  given  in  the  references  at  the  end  of 
this  Appendix,  so  the  following  discussion  will  be  somewhat 
general.  The  sample  size  calculation  process  should  be 
viewed  as  providing  input  for  the  broad  scoping  and  planning 
of  the  data  collection  effort.  The  specific  sample  size 
values  obtained  from  the  formulas  should  be  taken  as  rough 
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indications  of  lower  limits  for  data  collection,  rather  than 
as  precise  targets  or  cut-off  points.  Prudent  expansion 
factors  should  be  applied  to  the  calculated  sample  size 
values  so  that  the  ultimate  amount  of  usable  data  (i.e. , the 
net  sample  size  after  the  collection  activities  and  editing) 
is  sufficient  to  yield  results  with  the  desired  level  of 
precision  and  statistical  accuracy,  and  allows  for 
unforeseen  stratification.  As  data  is  collected,  it  should 
be  possible  to  modify  sample  requirements  for  subsequent 
phases  of  a project. 

As  has  been  mentioned  earlier,  it  is  desired  to  have 
results  presented  in  the  form  of  confidence  intervals. 
Determining  the  sample  size  for  calculating  a confidence 
interval  requires  three  input  factors: 

(1)  The  desired  confidence  level 

(2)  An  estimate  of  the  variablility  in  the  population 

(3)  The  desired  precision  of  the  results. 

The  confidence  level  of  a statistical  calculation  (1- 
a ) can  be  defined  as  the  proportion  of  samples  of  size  n 
for  which  the  calculated  confidence  interval  may  be  expected 
to  contain  the  true  value  of  the  population  parameter  being 
estimated.  For  purposes  of  obtaining  a conservative  sample 
size  estimate,  it  is  recommended  that  the  value  a =.05  be 
used. 


An  estimate  for  variability  is  usually  taken  as  the 
standard  deviation.  It  is  desirable  initially  for  this 
value  to  be  an  overestimate  to  allow  for  a conservative 
determination  of  sample  size.  While  it  is  preferable  to 
have  some  prior  knowledge  about  the  variability  of  those 
measures  to  be  collected,  Natrella  (pages  2-8  to  2-10)  gives 
an  excellent  approach  for  cases  where  the  true  standard 
deviation  is  unknown. 

Determination  of  an  acceptable  level  of  precision  is 
perhaps  the  most  difficult  input  factor.  In  the  case  of 
estimating  means,  variability  measures,  and  proportions,  the 
task  is  to  determine  the  acceptable  accuracy,  say  d,  for 
each  confidence  interval.  The  sample  size  calculated  on  the 
basis  of  a prescribed  d and  a = .05,  reflects  an 
acknowledged  (permissible)  risk  that  5 times  in  100  the  real 
precision  will  be  worse  than  d.  In  the  case  of  estimating 
the  difference  between  means  or  between  other  statistics, 
the  analogous  task  is  to  specify  the  absolute  value  of  a 
minimum  desired  detectable  average  difference  6.  Fere, 
too,  if  a = .05,  then  the  sample  size  will  reflect  an 
acknowledged  risk  that  5 out  of  100  times  the  true 
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difference  between  the  two  groups  being  compared  will 
exceed  6. 

In  establishing  values  for  d and  6,  consideration 
must  be  given  to  the  problem  of  trading  off  the  cost  vs. 
benefits  of  increased  precision.  The  cost  of  increased 
accuracy  can  be  seen  as  the  marginal  amount  of  time  and 
money  needed  to  collect  an  additional  sample  unit.  The 
benefits  of  increased  accuracy  can  be  viewed  in  terms  of 
additional  confidence  in  the  results  of  a particular  project 
and  the  consequent  willingness  of  UMTA  to  make  policy  and 
funding  recommendations  on  the  basis  of  these  results. 
Clearly,  UMTA  does  not  want  to  encourage  cities  to  implement 
service  and  methods  innovations  which  have  only  a negligible 
impact  on  the  quality  or  usage  of  transit  service;  this 
would  argue  in  favor  of  setting  relatively  large  values  of  d 
and  <5  . On  the  other  hand,  there  is  a desire  to  learn 
whatever  possible  about  the  effects  of  implementing  new 
techniques;  if  the  minimum  detectable  difference  is  set  too 
large,  the  resultant  sample  size  may  be  too  small  to  detect 
the  existence  of  minor,  possibly  unanticipated  changes  which 
might  be  of  interest.  Given  the  somewhat  different  emphasis 
of  exemplary  vs.  experimental  demonstrations,  a suggested 
approach  is  to  consider  using  smaller  values  of  d and  6 in 
the  latter  type  of  project,  where  the  nature  of  the  outcome 
is  less  certain. 

Working  with  the  cognizant  UMTA  and  TSC  professionals, 
the  contractor  should  indicate  the  value  of  d or  6 selected 
for  each  measure  to  be  collected,  and  should  explain  the 
rationale  for  choosing  the  particular  value  in  terms  of  the 
cost-benefit  considerations  discussed  above. 


4 . DATA  COLLECTION 

Cnee  the  minimum  sample  size  for  each  stratification 
category  of  each  sampled  measure  has  been  determined  using 
the  appropriate  formula  and  the  above  three  prescribed  input 
factors,  the  data  collection  phase  can  be  implemented.  As 
was  mentioned  above,  the  contractor  should  apply  a prudent 
expansion  factor  to  the  minimum  sample  size  to  obtain  a 
target  sample  size.  As  better  insight  is  obtained  on  the 
effectiveness  of  these  project  sampling  plans,  guidelines 
regarding  sample  expansion  factors  will  be  provided. 

Field  observations  should  be  scheduled  for  a sufficient 
number  of  days  to  collect  the  target  quantity  of  sample 
units.  In  most  cases,  the  scheduling  of  data  collection 
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will  present  no  particular  problems:  the  required  number  of 
"representative"  days  can  be  designated,  as  well  as 
alternate  dates  to  be  used  in  the  event  of  unusual  weather 
conditions  or  other  atypical  occurrences  on  the  planned 
dates.  However,  there  may  arise  a situation  where  the  day- 
to-day  variability  is  known  or  suspected  to  be  significant 
in  relation  to  the  variability  within  a day.  In  this  case, 
arbitrary  spreading  of  the  data  collection  phase  over 
several  consecutive  days  may  adversely  affect  the  inferences 
to  be  made.  Depending  upon  the  project  objectives,  it  may 
be  more  appropriate  to  schedule  data  collection  for 
consecutive  weeks  on  a particular  day  of  the  week  (the  most 
representative  day).* 


5.  ANALYSIS  METHODS 

Since  numerous  statistical  methods  are  available,  the 
balance  of  this  Appendix  discusses  a family  of  statistical 
techniques  which  will  be  appropriate  for  project  analyses. 
The  measures  which  will  be  collected  and  utilized  to  assess 
achievements  of  project  objectives  can  be  classified  as 
discrete  or  continuous.  A discrete  measure  is  one  which  can 
assume  only  a fixed  and  known  set  of  values.  Examples  of 
such  measures  would  be  counts  of  numbers  of  vehicles  and 
passengers,  responses  to  qualitative  questions  and 
classifications  of  survey  responses  into  categories  such  as 
yes/no.  Continuous  measures  may  assume  (in  theory)  an 
infinite  set  of  values.  The  accuracy  of  these  measures  is 
constrained  only  by  instruments  used  in  collecting  the  data 
and  the  errors  inherent  in  the  data  collection  methodology. 
Examples  of  continuous  measures  are  travel  time  and  vehicle 
speeds. 


*The  preceding  discussion  deals  with  day-to-day  variability 
with  a known  pattern.  In  the  unusual  situation  of  day-to- 
day  variability  which  exceeds  within-day  variability  and 
does  not  follow  a particular  pattern,  the  target  sample  size 
must  be  calculated  according  to  different  procedures,  which 
give  a number  of  sample  days  as  well  as  a number  of  samples 
per  day.  A.  Ha Id,  Statistical  Theory  with  Engineering 
Applications,  John  Wiley  and  Sons,  Inc.,  New  York,  1952, 
Section  17.4  gives  procedures  which  could  be  used  in  dealing 
with  this  situation. 
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Depending  on  the  type  of  measure  being  collected,  one 
or  more  of  the  following  statistics  will  be  obtained: 

(1)  Averages  (mean  values) 

(2)  Standard  deviations  (variances) 

(3)  Ratios,  proportions 

(*»)  Ranges  for  the  raw  data 

(5)  Frequency  distributions  of  the  raw  data. 

In  addition  to  these  five  basic  statistics,  recent 
experience  on  several  demonstration  projects  indicates  the 
importance  of  the  statistical  measure  known  as  the 
coefficient  of  variation,  namely,  the  ratio  of  the  standard 
deviation  to  the  arithmetic  mean.  The  contractor  should  be 
alert  to  the  potential  use  of  other  statistical  measures  in 
his  analysis  of  project  data. 

Confidence  intervals  will  be  computed  for  differences 
between  means  and  proportions  and  for  ratios  of  variability 
measures.  The  procedures  for  calculating  confidence 
intervals  on  ratios  of  means  and  other  ratios  will  not  be 
given  here,  due  to  the  complexity  of  the  mathematical 
formulas. 

Actual  calculations  of  confidence  intervals  depend 
usually  on  four  elements:  the  sample  statistic  being  used  to 
estimate  the  population  parameter  (defined  above) , some 
measure  of  variability  associated  with  this  statistic  (e.g., 
the  sample  standard  deviation) , the  confidence  level,  and 
the  sample  size. 

Commonly  used  confidence  levels  have  99%  and  95% 
probabilities  associated  with  them.  These  correspond 
to  a=.01  and  a=.05.  It  is  recommended  that  the 
contractor  compute  and  report  confidence  interval  estimates 
based  on  both  values  of  a.  This  allows  the  decision-maker 
to  assess  both  intervals  and  to  determine  which  risk  level 
is  acceptable.  (Note:  For  an  a=.05,  while  there  is  a 95% 
chance  that  the  method  employed  will  contain  the  true  value 
of  the  parameter  being  estimated,  there  is  also  a 5%  chance 
that  the  intervals  will  not  contain  this  true  value).* 


*It  should  be  noted  that  while  the  use  of  confidence 
intervals  is  required,  the  contractor  may  apply  statistical 
tests  of  significance,  where  appropriate. 
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It  should  be  noted  that  the  sample  size,  n,  which 
should  be  used  in  computing  confidence  intervals  is  the 
actual  number  of  sample  observations  made,  which,  in  most 
cases,  will  be  different  from  the  number  originally  planned. 

Appropriate  methods  of  analysis  are  now  described  in 
terms  of  discrete  and  continuous  measures.  It  is  implicit 
in  any  analyses  performed  using  inferential  statistical 
methods  that  the  reasonableness  of  assumptions  will  be 
tested,  for  example,  normality.  If  the  data  being  collected 
can  be  classified  as  discrete,  the  following  techniques  may 
be  used: 

(1)  Confidence  intervals  on  a sample  proportion  to 
estimate  the  true  population  proportion.  The 
appropriate  techniques  here  will  be  to  use  either 
the  binomial  distribution  or  the  normal 
distribution,  depending  primarily  upon  the  sample 
size. 

(2)  Confidence  intervals  on  differences  between  two 
proportions.  In  this  situation  the  appropriate 
methodology  is  again  to  use  the  binomial 
distribution  or  normal  distribution,  depending  on 
sample  size.* 

If  the  data  element  being  collected  during  the 
demonstration  can  be  classified  as  continuous,  then 
appropriate  methodologies  which  can  be  used  are: 

(1)  Establishing  confidence  intervals  on  sample  mean 
values  to  estimate  population  mean  values.  The 
appropriate  methodology  will  involve  Student's  "t" 
distribution . 

(2)  Establishing  confidence  intervals  on  sample  mean 
differences.  The  appropriate  methodology  will  be 
to  use  the  Student's  "t"  distribution. 

(3)  Determining  whether  differences  observed  from  more 
than  two  sample  mean  values  can  be  classified  as 
significant.  The  appropriate  methodology  here 
would  involve  use  of  the  F distribution  and  the 


*When  appropriate,  other  methods,  such  as  chi-square,  may 
be  used  to  assess  significance  of  differences  in  discrete 
classifications  where  there  are  more  than  two  alternatives. 
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analysis  of  variance,  coupled  with  the  application 
of  appropriate  linear  contrasts  techniques.  (See 
C.H.  McCall,  "Linear  Contrasts",  Parts  I,  II,  and 
III,  Industrial  Quality  control.  Volumes  XVII, 

Nos.  1,2,  and  3,  July,  Aug.,  Sept.,  1960.) 

(4)  Establishing  confidence  intervals  on  a single 
variance.  The  appropriate  methodology  will  be 
chi-square. 

(5)  Establishing  confidence  intervals  on  ratios  of 
variances.  The  appropriate  methodology  will  be 
the  F distribution. * 


6.  METHODOLOGY  DOCUMENTATION 

The  contractor  shall  document  and  explain  all 
considerations  in  data  analysis  and  sample  size  selection 
for  each  measure  including: 

(1)  How  variability  was  estimated. 

(2)  Pationale  for  the  desired  level  of  precision 
chosen. 

(3)  How  the  final  sampling  plan  was  established  to 
ensure  that  an  adequate  sample  size  would  be 
available  for  analysis. 

In  addition,  the  method  planned  for  performing  all 
statistical  calculations  and  tests  should  be  documented  by 
reference  to  the  appropriate  equations  and  tables  in 
Natrella  or  other  reliable  sources. 


♦For  more  than  two  variances,  tests  of  significance  rather 
than  estimating  confidence  intervals  may  be  appropriate. 
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APPENDIX  C 


GLOSSARY 


Access  Point  --  That  point  at  which  a transit  vehicle  can  be 
boarded. 


Access  Time  — The  time  from  leaving  a point  of  origin  to 
arriving  at  a point  where  an  element  of  the  transit  system 
can  be  boarded. 


Arterial  — A highway  primarily  for  through  traffic,  usually 
on  a continuous  route. 


Central  Business  District  (CBD)  --  Usually  the  downtown 
retail  trade  area  of  a city  with  a concentration  of  retail 
business  offices,  theaters,  hotels,  and  service  businesses. 
Generally  an  area  of  very  high  land  valuation  and  heavy 
traffic  flow. 


Collected  Measure  — A data  element  which  is  directly 
collected,  either  by  measurement  (e.g.,  vehicle  travel  time 
in  minutes)  or  counting  (e.g.,  number  of  vehicles). 


Continuous  Measure  — A data  element  which  can,  in  theory, 
assume  an  infinite  number  of  alternative  values  (e.g., 
travel  times,  vehicle  speeds,  distances) . The  accuracy  of 
the  recorded  values  for  these  measures  is  dependent, 
primarily,  upon  the  instrument  being  used  to  collect  the 
data. 


Corridor  --  A route  or  group  of  routes  having  similar  travel 
characteristics  and  generally  emanating  from  the  CBD. 
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Derived  Measure  — A data  element  which  is  calculated  from 
basic  measures  (e.g.,  passenger  miles  per  revenue  mile). 


Destination  — Terminal  end  of  a trip  or  the  point  at  which 
a trip  terminates. 


Discrete  Measure  — A data  element  which  can  assume  only  a 
fixed  number  of  alternative  values  (e.g.,  a yes/no  response; 
classification  by  mode  of  travel) . 


Egress  Point  --  That  point  at  which  the  passenger  leaves  the 
last  transit  vehicle  to  be  used  in  going  from  his  origin  to 
his  destination. 


Egress  Time  --  The  time  it  takes  after  leaving  the  egress 
point  to  arrive  at  the  destination. 


Elderly  --  Generally  accepted  for  evaluation  purposes  as 
persons  65  or  over.  It  is  noted,  however,  that  the  term 
elderly,  or  also  seniors,  is  often  applied  to  ages  as  low  as 
60,  sometimes  55. 


Evaluation  Plan  — An  evaluation  plan  sets  the  demonstration 
project  frame  of  reference  and,  through  project  objectives, 
identifies  the  ways  in  which  necessary  data  elements  will  be 
collected,  processed,  summarized,  analyzed,  and  interpreted 
(it  is  recognized  that  modifications  may  be  necessary  as  a 
project  develops) . 


Experimental  Design  — Similar  to  an  evaluation  plan,  except 
that  the  data  elements  are  obtained  through  a well 
controlled  experimental  situation.  At  UMTA,  the  phrase 
"experimental  design  plan"  is  sometimes  used  interchangeably 
with  evaluation  plan. 
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Handicapped  --  A person  who  by  reason  of  illness,  injury, 
age,  congenital  malfunction,  or  other  permanent  or  temporary 
incapacity  or  disability,  is  unable  without  special 
facilities  or  special  planning  or  design  to  utilize  mass 
transportation  facilities  as  effectively  as  persons  who  are 
not  so  affected. 


Headway  — The  time  lag  between  transit  vehicles  moving  in 
the  same  direction  on  any  given  route. 


Innovation  — An  event  or  activity  which  is  introduced 
within  the  project  service  area  for  purposes  of  assessing 
its  effect  on  the  transportation  system  and  its  impact  on 
the  community  being  served. 


Low  Mobility  Groups  --  Those  who  because  of  lack  of 
opportunity  or  ability  to  use  automobiles,  or  because  the 
absence  of  adequate  public  transportation,  or  because  of  the 
lack  of  motivation  or  need,  travel  considerably  less  than 
others.  Included  are  all  of  the  transit  dependent  groups 
except,  possibly,  youth. 


Measure  --  A data  element  to  be  obtained  during  a 
demonstration  project  for  purposes  of  evaluating  project 
objectives. 


Miles  of  Transit  Route/ Arterial  Lanes  --  Total  route  miles 
for  transit  system.  Total  lane  miles  of  highway  system. 


Modal  Split  --  The  separation  of  total  person  trips  into 
various  modes  of  travel. 
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Near  Poor  --  The  term  "near  poor"  refers  to  families  just 
above  the  poverty  line,  including  non-farm  families.  The 
near  poor  designation  refers  to  families  with  incomes  up  to 
one-third  above  the  incomes  cited  under  the  definition  of 
poor.  A near  poor  family  of  three  would  have  an  annual 
income  of  $4,230  to  $5,640. 


Origin  — The  beginning  of  a trip  or  the  zone  in  which  a 
trip  begins. 


Passenger  Trip  — The  movement  of  a person  in  a vehicle 
between  two  points  separated  in  space  for  a purpose  other 
than  solely  continuing  that  movement. 


Peak  Hour  --  That  hour  period  during  which  the  maximum 
amount  of  travel  occurs  (e.g.,  a morning  and  afternoon 
peak)  . 


Peak  Period  --  That  time  period,  usually  longer  than  an 
hour,  during  which  the  maximum  amount  of  travel  occurs 
(e.g.,  an  A.M.  and  a P.  M.  peak). 


Poor  — A poor  person  is  one  who  is  a member  of  a poor 
family.  The  definition  of  a poor  or  "economically 
disadvantaged  family"  as  defined  by  the  U.S.  Department  of 
Labor  is: 


Family  Size 
1 
2 

3 

4 

5 

6 


Annual  Income 
$2590 
3410 
4230 
5050 
5870 
6690 


For  each  additional  family  member  add  $820 


Because  of  differing  transportation  needs,  a distinction  is 
made  between  poor  persons  in  the  labor  force  and  outside  the 
labor  force. 
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Project  Service  Area  --  That  geographic  region  from  which 
potential  customers  for  the  transit  system  can  be  drawn. 


Revenue  Miles  --  Sum,  for  each  vehicle  type  in  a 
transportation  system,  of  the  mileage  over  which  revenue  can 
be  generated. 


Route  --  A fixed  path  traversed  by  a transit  vehicle  in 
accordance  with  a pre-determined  schedule. 


Run  --  One  transit  vehicle  trip  in  one  direction  from  the 
beginning  of  a route  to  the  end  of  it.  When  a transit 
vehicle  makes  a round  trip  on  one  route,  it  has  completed 
two  runs. 


Screenline  --  An  imaginary  line  dividing  the  study  area 
(demonstration  area)  into  two  parts  for  purposes  of 
analysis . 


Seating  Capacity  — Total  number  of  seats  available  on  an 
operating  transit  vehicle. 


Transit  Dependent  Persons  --  Those  who  because  of  age, 
income,  or  physical/mental  incapabilities  must  rely  on 
public  transportation,  e.g.,  do  not  have  use  of  automobiles 
except  as  passengers.  Included  are  the  elderly, 
handicapped,  youth  and  poor  (unemployed  as  well  as  non- 
members of  the  labor  force) . 


Trip  — A person  or  vehicle  movement  which  begins  at  the 
origin  at  the  start  time,  and  ends  at  the  destination  at  the 
arrival  time  and  is  conducted  for  a specific  purpose. 


Unemployed  — Persons  who  are  members  of  the  labor  force  and 
who  are  registered  as  seeking  a job.  The  labor  force  is  the 
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sum  total  of  all  employed  persons  plus  all  persons 
registered  as  unemployed.  The  unemployed  are  further 
separated  into  frictional  unemployed  (persons  who  are  in  the 
process  of  changing  jobs  where  such  jobs  are  available) , 
demand  unemployed  (the  excess  of  trained  workers  over 
available  jobs)  , and  chronic  or  hard-core  unemployed 
(individuals  who  have  been  unemployed  for  16  weeks  or  more 
and  who  have  obsolete  or  non-usable  skills  or  who  are 
systematically  precluded  from  employment  because  of  some 
feature  of  their  being,  e.g.,  their  age,  physical 
capability,  language  capability,  etc.) . 


Vehicle  Count  --  The  total  number  of  vehicles  in  operation 
as  detected  by  a vehicle  count  for  each  category. 


Vehicle  Fleet  --  The  total  number  of  vehicles  owned  or  being 
used  under  a purchase  lease,  related  parties  lease,  or  a 
true  lease. 


Vehicle  Miles  --  Sum  for  each  vehicle  type  in  a 
transportation  system  of  the  total  mileage  placed  on  the 
vehicle  type  during  the  reporting  period.  Can  be  classified 
into  in-service  (revenue)  and  non-service  (non-revenue) 
vehicle  miles. 


Vehicle  Trip  — A vehicle  movement  which  begins  at  a 
specific  start  point  and  ends  at  a specific  destination, 
said  trip  being  for  the  purposes  of  revenue  generation  (see 
trip)  . 


Youth  --  For  evaluation  purposes  persons  who  are  17  or 
under,  based  on  the  trend  of  state  legislatures  to  designate 
age  18  as  the  age  of  adulthood  or  majority.  Most 
evaluations  will  deal  with  the  subgroup  of  ages  9 through  15 
(and  may  refer  to  this  subgroup  as  "youth") . The  lower 
limit  is  an  age  when  independent  mobility,  without  parents, 
is  expected.  The  upper  limit  is  the  earliest  age  when  a 
driver's  license  is  allowed. 
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Bus  -on-  Freeway  Demonstration  Project  - Second  Year 
Results , Interim  Report  4 (Report  DOT/UMTA  4)  , U.S. 
Dept,  of  Commerce,  National  Bureau  of  Standards, 
November,  1973. 
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"Morgantown  PRT  Impact  Evaluation  Plan,"  Morgantown,  West 
Virginia,  Revised:  October  15,  1974. 


Pha se  I , Data  Summary  Memorandum,  I-35W  Urban  Corridor 

Demonstration  Project,  Metropolitan  Council,  St.  Paul, 
Minnesota,  November,  1974. 


Phase  2_  Evaluation  Results , I-35W  Urban  Corridor 

Demonstr at  ion  Project,  Metropolitan  Council,  St.  Paul, 
Minnesota,  November,  1974. 


(Phase  3)  I-35W  Urban  Corridor  Demonstration  Project  - Bus- 
Mete  red  Freeway  System,  Final  Report,  Metropolitan 
Council,  St.  Paul,  Minnesota,  August,  1975. 


Project  Fare  Tast  IV  Report;  Urban  Mass  Transportation 

Industry  Financial  and  Operating  Data  Reporting  System, 
Vol.  1 - Task  and  Project  Summary,  Arthur  Anderson  and 
Co.,  Washington,  D.C.,  November,  1973. 


Sta tus  of  the  Urban  Corridor  Demonstration  Program,  July, 
J_97i4.  (DOT  P 65002)  United  States  Dept,  of 
Transportation,  Washington,  D. C. , 1974. 


Transbus  Public  Testing  and  Evaluation  Program;  Procedures 
for  Evalua ti on  by  General  Public  and  Transit  Operating 
Personnel,  Simpson  and  Curtin,  Philadelphia, 
Pennsylvania,  February,  1973. 


Urban  Corridor  Demonstration  Program,  (DOT  P 6500.1),  United 
States  Dept,  of  Transportation,  Washington,  D.C., 
October,  1974. 


Urban  Corridor  Demonstration  Program  Evaluation  Manual, 
Final  Draft,  (DOT-OS-10029) , Texas  Transportation 
Institute,  Texas  A & M University,  College  Station, 
Texas,  August,  1971. 


Urban  Mass  Transportation  Travel  Surveys , United  States 
Dept,  of  Transportation,  Washington,  D.C.,  August, 
1972. 
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Waksman,  Robert,  The  Shirley  Highway  Express- Bus- On- Freeway 
Demonstr ation  Project/A  Study  of  Reverse  Commute 
Service,  Interim  Report  5 (NBSIR  74-624),  U.S.  Dept,  of 
Commerce,  National  Bureau  of  Standards,  December,  1974. 


Miscellaneous  UMTA  Documents: 


"Experimental  Design  - The  Bus  Repair  and  Overhaul 
Automatic  Diagnostic  Systems  Development" 


"Experimental  Design  Package  (XDP)  for  the  Downtown 
Circulation  System" 


"Experimental  Design  Package  (XDP)  for  the  I-95/N.W. 
7th  Avenue  Bus/Carpool  Systems  Demonstration 
Pro ject (s) " 


"Experimental  Design  Package  (XDP)  for  the  Modern 
Passenger  Transit  Bus  Prototypes  Project  (1,100)" 


"Review  of  Five  Rail  Demonstration  Experimental  Design 
Packages " : 

(1)  The  Stored  Energy  Rapid  Rail  Car  (Flywheel) 

(2)  The  Cleveland  Transit  System/WABCO  DC  to  AC 
Propulsion  Demonstration  Project 

(3)  The  GT-IV  Gas  Turbine/ Electric  Commuter  Car 
Demonstration  Project 

(4)  The  Light  Rail  Vehicle  Demonstration  Project 

(5)  The  Rail  Bus  Demonstration  Project 


"The  UMTA/ Transit  Performance  Measure  Framework" 
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